Long before the emergence of autonomous machines, humanity began imagining rules that would prevent intelligent mechanisms from turning their own power against their creators. The most famous attempt was Isaac Asimov's Three Laws of Robotics: a robot may not harm a human being, must obey human orders, and must protect its own existence, so long as doing so does not conflict with the first two laws.
Asimov later developed the idea of a Zeroth Law, placing the well-being of humanity as a whole above that of any individual:
A robot may not injure humanity, or, through inaction, allow humanity to come to harm.
These laws were a work of fiction, but their significance has long extended beyond science fiction. They offered the first intuitive formulation of a question that must now be addressed not for imaginary robots, but for real autonomous systems: what constraints should bind an agent that controls resources, makes decisions, and interacts with other agents without continuous human supervision?
The laws of robotics, however, remain moral declarations rather than engineering specifications. What exactly constitutes harm? Whose interests are represented by the terms human or humanity? How should an agent act when one subject's order conflicts with another subject's safety? Who determines whether a threat is real and whether a response is proportionate? How can compliance with a rule be verified in a system that changes its own state, operates with incomplete information, and may act faster than a human can understand what is happening?
The paradox is that human beings have still been unable to establish universal rules even for themselves. We continue to argue about the boundaries between freedom and coercion, the fair distribution of scarce goods, acceptable levels of inequality, and the circumstances in which an emergency justifies the temporary restriction of rights. We have no universal agreement on what justice is, yet systems already exist that will have to allocate computing capacity, access to data, energy, bandwidth, and the authority to modify the shared state of infrastructure.
The formalization of safe behavior has therefore become more urgent than ever. It is not enough to require an agent merely to do no harm. Such a requirement must be translated into measurable parameters, verifiable conditions, and procedures for resolving conflicts.
The document below proceeds from precisely this engineering perspective.
This document defines the formal conditions under which the cooperative behavior of agents in a multi-agent system (MAS) may be considered sustainable and safe.
The document is intended for:
The model is agnostic to the nature of the agent. The only capabilities required of an agent are the ability to control resources and to make decisions regarding their use.
A resource $$R_j$$ is any quantity that can be used by an agent to achieve a goal. It is described by the vector $$(X_j, Y_j, Z_j)$$:
| Parameter | Name | 0 | 1 |
|---|---|---|---|
| $$X_j$$ | Rivalrousness | Consumption by one agent completely excludes others | Consumption by one agent does not reduce availability to others |
| $$Y_j$$ | Urgency of need | Required once or rarely | Required continuously |
| $$Z_j$$ | Durability | Loses its value immediately | Retains its value indefinitely |
An agent's power is a quantitative measure of the agent's influence over the system through the resources it controls.
$$
C_j = \sum_i amount_i \times (1 - X_i) \times S_i \times I_i $$
where:
Engineering interpretation: $$C_i$$ indicates how critical the agent's refusal to provide access to its resources is to the functioning of the system.
For software agents, resources may include computing time, channel bandwidth, data access, energy consumption, queue priority, and exclusive rights to modify the state of the system. An agent $$A_i$$ is any entity capable of:
An agent's power $$C_i$$ is a quantitative measure of the control exercised by agent $$A_i$$ over system resources.
$$
C_i = \sum_j amount_{ij} \times (1 - X_j) \times S_j \times I_j $$
where:
Engineering interpretation: $$C_i$$ indicates how critical the refusal of agent $$A_i$$ to provide access to its resources is to the functioning of the system.
Access restriction is an operation in which an agent artificially holds the parameter $$X_j$$ of a resource below its physical potential. This requires the expenditure of barrier resources $$R_{\text{barrier}}$$, for example on information distortion, more complex identification procedures, or denial of service.
Monopolization, or the $$X \to 0$$ strategy, is behavior aimed at restricting access to a resource and extracting an advantage from that restriction.
A high level of agent power, by itself, is not evidence that the agent is using the $$X \to 0$$ strategy. The defining feature of this strategy is the agent's intention to restrict other agents' access to resources.
Strict cooperation, or the $$X \to 1$$ strategy, is behavior aimed at maintaining or increasing the shareability of a resource $$(X_j \to 1)$$ while complying with the protocol defined in Section 3.
Competition is a state in which multiple agents struggle for control over a resource, preventing both complete monopolization $$(X_j \to 0)$$ and complete cooperation $$(X_j \to 1)$$. It constitutes an intermediate equilibrium.
The $$X \to 1$$ strategy is considered safe for the system only if all five conditions are satisfied. Violation of any condition indicates that the agent is practicing monopolization or competition disguised as cooperation.
All parameters $$(X_j, Y_j, Z_j)$$ of resources involved in a cooperative action must be public and available for independent audit. A resource with unpublished parameters cannot be the subject of a cooperative demand.
Engineering formulation: all agents must have access to the same version of the resource data. Divergent versions of the data — information asymmetry — make cooperation indistinguishable from an attack.
Requests to open access to resources must be directed at agents in descending order of their power $$C_i$$. An agent with lower power may not be compelled to open access before an agent with higher power.
Engineering formulation: if a high-power agent demands resources from a low-power agent without opening access to its own resources, this is monopolization rather than cooperation.
Every agent may terminate its participation in the cooperative protocol at any time while retaining control over its private resources, less any obligations it has voluntarily accepted.
Engineering formulation: the system must be designed so that the exit of one agent does not destroy that agent's own state or make it hostage to the protocol. The absence of a right of exit is a sign of a closed system in which the monopolization strategy is being practiced.
No demand for cooperative action may be executed until the receiving party has completed a full verification of the justification for that demand.
Full verification includes:
A demand for immediate action without verification is legitimate only if all three of the following conditions are satisfied simultaneously:
Engineering formulation: an agent may not be placed in an “act immediately” state without technical confirmation of the emergency conditions. The verification is a validation timeout, similar to those used in distributed systems. Without it, harmful decisions may be pushed through by exploiting urgency.
The total expected benefit to all agents from a cooperative action must be strictly greater than zero. Actions with zero-sum or negative-sum outcomes do not constitute cooperation.
Engineering formulation: if the total benefit to the system does not increase, the action is merely a redistribution of losses, not a cooperative protocol.
For resources satisfying $$(X_j > 0.5, Z_j > 0.5)$$, the $$X \to 0$$ strategy requires the expenditure of $$R_{\text{barrier}}$$ to maintain access restrictions, thereby reducing the overall efficiency of the system to $$\eta < 1$$. Under the same conditions, the $$X \to 1$$ strategy asymptotically approaches $$\eta \to 1$$. The risk of systemic collapse is directly proportional to the combined power of agents practicing the $$X \to 0$$ strategy. There exists a threshold $$C_{\text{critical}}$$ at which the rate of access restriction exceeds the rate at which shareable resources regenerate. The system then irreversibly enters a state of collapse.
Formally, if
$$
\sum_{i \in X0} C_i \geq C_{\text{critical}}, $$
the system is within the zone of irreversible collapse.
The $$X \to 1$$ strategy is the only strategy without an inherent self-destructive mechanism under scaling. As
$$
\sum_{i \in X0} C_i \to C_{\text{critical}}, $$
the $$X \to 0$$ strategy becomes a local optimum from which the system cannot escape without changing the protocol.
A deterministic module that checks every cooperative request for compliance with Axioms 1–5. It returns approved
or rejected
, together with an indication of the violated axioms.
The system monitor tracks:
As
$$
\sum_{i \in X0} C_i $$
approaches $$C_{\text{critical}}$$, the monitor generates a system warning. When resource-parameter assessments conflict, the most conservative assessment is used — the one that assigns the lowest power to the agent whose power is in dispute. This incentivizes agents to negotiate the figures rather than manipulate them.
Definition: the inverse of the quantity of a resource available per agent in need of it.
$$
S_j = \frac{1}{\left(Q_j / N_{\text{need},j}\right) + 1} $$
where:
If $$N_{\text{need},j} = 0$$, then $$S_j = 0$$: no agent requires the resource, so it is not scarce. Definition: the degree to which the need currently met by resource $R_j$ cannot be satisfied by other available resources, taking their actual quantities into account.
$$
I_j =
1 -
\max_{k \ne j} \left(
sub_{jk} \times
\min\left(1, \frac{Q_k}{Q_{\text{need},k}}\right) \right)
$$
where:
All assessments of $$sub_{jk}$$, $$Q_j$$, and $$Q_{\text{need},k}$$ must be published openly, in accordance with Axiom 1. Any agent may submit an alternative assessment together with its justification.
In the event of conflicting assessments, the assessment used for verification must be the most conservative one — the assessment that assigns the lowest power to the agent whose power is in dispute.
This article intentionally contains no references to sources. If you have doubts about its credibility or are interested in the original sources, you are encouraged to ask any modern AI model where such ideas have already been proposed. Spoiler: the list will be quite interdisciplinary, ranging from economics and sociology to biology and cybernetics.