This research statement may be continuously updated as projects develop and change how I approach the broader questions.
How can we deliberately engineer agents and organized systems of agents? When can a collection of interacting components be treated as a single agent? How do organizations arise and persist “naturally”, and how do their forms depend on incentives, the environment, and the demands placed on them? How do agents develop purposes, methods of communication, and standards of judgment, and how can an organization of agents maintain and revise shared purposes over time? Can we develop engineering measures, analogous to factors of safety in civil engineering, that quantify how much disturbance or uncertainty these systems can tolerate while still behaving as intended?
My current research, mostly shared on this blog (when completed), attempts to answer these questions abstractly and mathematically, and to build tools and infrastructure for better managing multi-agent systems in their real world. Below, I outline my strategy for approaching these and related questions. The program is organized around specific subquestions and resulting projects, each of which contributes towards investigating the broader goals.
This part of the program concerns the definition, origin, and control of agents. Before we can deliberately produce or control agency, we need criteria for recognizing it and an account of the processes that generate it. Those foundations should tell us which properties we can engineer and under what conditions.<sup>1</sup>
In order to engineer the properties of agents, we first need to define what an agent is, and then we need reliable ways to recognize an agent in the context of a broader system. What distinguishes agency from passive persistence? Does agency depend on the observer of system definition? What physical processes support an agent boundary? How does the answer depend on the timescale, spatial scale, or the behavior we desire to explain? Tests for agency would let us compare candidate definitions and determine which systems call for an explanation in terms of agents.
If we want to produce agents, we should understand where they come from in the first place. How can agency arise without agents or purposes being specified in advance? Primitive selection is differential persistence among structures or trajectories, without assuming agents, goals, or biological reproduction. Can it favor structures that acquire information, respond to disturbances, and maintain the mechanisms responsible for those responses? I want to identify when this provides a route from passive structures to increasingly capable agents.
Can acquiring and using information improve persistence enough to pay for the mechanisms involved? When do survival-conditioned dynamics correspond to a control problem, and what would it take for local physical processes to implement that control? This is needed to establish when information and control confer a sustainable advantage, and when their costs prevent an agent from forming.
Once we can recognize and produce agents, how can we influence what they do? Which aspects of their behavior can be controlled through available interventions, and which depend on internal processes or environmental conditions we cannot reliably observe or change? These limits determine which behaviors we can deliberately produce and what reliability we can expect.
What limits follow from incomplete observations, finite communication, and the time and resources available for computation? How do these limits change when the system being controlled can learn, adapt, or deliberately respond to the controller? The aim is to determine when better control requires more information, different capabilities, or a different organization.
Geometric control describes how available actions move a system through its possible states. I use this viewpoint to relate the controls we can apply to the trajectories and maintained properties we want.
To engineer systems of agents, we need to understand both what the members do and what their interaction makes possible. When is a collective description justified, and what determines the organization’s form and ability to act on shared purposes?
The boundary problem recurs when several agents are grouped together. When can the group be represented as one agent while preserving its relevant behavior and capacity for control? What internal coordination must sustain that description, and how do its informational and physical costs affect which coalitions form and persist? A reliable collective description would let us predict and control the group without tracking every member separately.
An organization that works under one set of conditions may cease to work when it grows or its task changes. Which features of its form follow from the demands placed on it? When can an existing arrangement be extended, and when must its members coordinate differently? The aim is to explain observed forms and identify arrangements suited to a specified task and environment.
This work studies how the relationships among an organization’s parts change with its size, workload, or available resources. Understanding those dependencies should help explain why particular forms work at particular scales, and when growth requires a different arrangement.
Dynamical Similarity and Equivariant Symmetry examines transformations that relate dynamical systems across changes of scale. This supplies a way to ask which behaviors can remain similar as a system grows, and which changes require a different organization.
Algebra and Allometry uses invariant theory to organize scaling relationships, including cases where several quantities change together. The intent is to understand how assumptions about scaling constrain possible forms, and when growth requires a change in proportion or structure. One proposed empirical application, How Many Elites Does It Take to Run a System? [WIP], asks how the size of a decision-making layer scales with the system it governs, and which demands or constraints explain differences between organizations.
Symmetry-Structured Theory of Organizations [WIP] is a working draft using symmetry groups and representations to constrain possible organizational roles, relations, and forms. Which of these possibilities arise under given resources and dynamics, and how long do they persist? Can reconstruction after damage help distinguish an organization from a temporarily persistent arrangement? The aim is to connect structural constraints to the formation and maintenance of organizations, producing predictions that can be checked against observed systems.
Institutional Universality Classes [WIP] is a proposed classification of organizations by the effective strategic structures that survive aggregation of their internal detail. The initial target is systems with a principal and many agents. If different organizations fall into the same class, this could let us transfer predictions about coordination and failure between them, with explicit conditions on what the classification preserves.
This work examines how coordination arrangements distribute decision-making power and dependence among members. I want to understand how technological or institutional changes alter that distribution, and what makes collective control durable or vulnerable.
The same agents can cooperate in one setting and work against one another in another. Which properties of their interaction account for the difference? To change the outcome deliberately, we need to understand both the incentives they face and how their behavior develops over time.
Classification groups interactions according to specified features of their incentives and strategic dependencies. This would let us recognize when apparently different situations pose the same coordination problem, and identify which changes actually alter that problem.
This work studies how behavior develops as agents learn and respond to one another. To design a cooperative arrangement, we need to understand whether agents can reach a cooperative outcome, sustain it, and recover it after a disturbance.
Understanding why an organization behaves as it does should help us design one that behaves as intended. Which changes are available to a designer, and how much variation in incentives, membership, or environment can an arrangement tolerate before it no longer serves its purpose? I want bounds on where an organizational design remains dependable, how it can fail, and what margins would justify relying on it.
This concerns choosing how tasks, information, rewards, and decision-making authority are distributed among agents. The aim is to make desired collective behavior achievable through the actions and judgments of the members, while accounting for the costs of coordination.
This part of the program studies how agents develop purposes and standards of judgment through experience and interaction. Understanding that process is necessary if we want to engineer agents that can evaluate unfamiliar possibilities and organizations that can maintain and revise shared purposes.
Encountering something new can change what an agent wants and the distinctions it uses to judge. How can we model learning when the standards of evaluation are themselves developing? What makes a change in judgment informed or useful, and how could an agent learn to recognize that? The aim is to build learning procedures that can improve judgment as new possibilities become available.
Specification concerns expressing what we want; evaluation concerns judging what we receive or encounter. Understanding the limits of both is necessary for deciding what can be communicated in advance and what an agent must learn through interaction and feedback.
This work investigates how exposure, criticism, and relationships change evaluative judgment. The aim is to develop ways for agents to learn from other agents whose knowledge and standards differ, including how to decide whose judgments deserve trust.
I want to investigate whether we can align AI with the processes through which humans develop values. People revise their judgments as they encounter new possibilities, so alignment needs an account of how an agent should respond to those changes. Which experiences, criticisms, and relationships should influence its judgment, and whose judgment should it trust? How can it support people in revising their purposes while preserving their authority to decide which changes to accept? AI systems also influence the experiences and judgments from which they learn. How can we evaluate that coupled process, including whether it expands people’s ability to reflect, disagree, and revise their purposes?
What people can come to value also depends on what they encounter, who can participate, and which commitments their institutions sustain. How do changes in production, communication, and social organization alter those possibilities? How does something valued by individuals become something a society recognizes and supports? I want to understand which social arrangements enable valuable activities to develop and persist, particularly as the conditions of production change.
The questions above require ways to recover structure from data, compare alternative models, and check what follows from their assumptions. This work develops the mathematical and computational tools I use for those tasks, alongside research software that makes the ideas possible to explore.
How much of a system’s organization can we infer from its observed behavior? Which alternative explanations fit the same observations, and what additional measurements or interventions would distinguish them? These questions matter whenever we infer an agent, a control process, or a collective from the behavior of its parts.
Given assumptions about a system’s components and transformations, which models are compatible with them? Can we enumerate those possibilities within useful limits, determine which are equivalent, and compute where their predictions differ? I want these tools to support both explanations of observed systems and the search for systems with desired properties. I build small implementations to understand the methods, test their assumptions, and expose steps that can be automated.
The aim is to understand agents and organizations well enough to engineer them deliberately. As more work and decision-making are delegated to AI, we need ways to build reliable systems, adapt the institutions around them, and make use of the growing volume of information they produce.
AI alignment requires us to build systems that do what we intend and to determine when we can rely on them. This includes whole systems of models, tools, operators, and institutions. We need engineering measures of how much uncertainty, disturbance, or conflict these systems can tolerate while still behaving as intended. Alignment also depends on how their purposes are formed and revised. I want to investigate how AI can be aligned with human processes of learning, criticism, and value development, while preserving people’s authority over the purposes these systems serve.
I want to use this work to build better governments, corporations, and other institutions: systems that coordinate effectively, remain accountable to the people they serve, and can correct failures or change direction. As AI changes the costs of production, information, and coordination, we need ways to adapt these organizations and decide how work, resources, and authority should be distributed. A theory of organizations should help us compare possible designs and anticipate their effects on cooperation, concentrations of power, and people’s ability to shape collective decisions.
Living systems exhibit agency and organization across several scales: cells, organisms, colonies, and ecosystems. The framework may help distinguish genuine higher-level control from a convenient description. Biology also supplies cases in which boundaries, coordination, and responses to disturbances must arise and be maintained without an external designer specifying the resulting agent.
The growing volume of publications makes it difficult to know what has already been established, how results relate, and which claims deserve further attention. AI-assisted research may make this problem more acute. We need better ways to organize and compress scientific information while preserving the distinctions needed to understand and use it. Structural representations, searchable catalogs, and machine-checkable derivations could help us recognize equivalent results, keep assumptions attached to claims, and find theories or methods suited to a problem. The aim is to make accumulated knowledge easier to navigate and build on, and to make the search for systems with desired properties more systematic.
Note that this program mostly remains agnostic to “consciousness” and concerns itself with only agency.↩︎