A new paper was released at ICML that I'm worried will open an entire new dimension of alignment problems:
Latent Collaboration in Multi-Agent Systems (LatentMAS) TLDR: they show that multiagent systems can communicate faster and more efficiently by directly sharing latent states instead of text. This is a large performance improvement that comes at the cost of interpretability.
Previously, the handover between agents was a bottleneck where latent thoughts had to be put into text. Steganography was the only way that models could pass along misaligned plans between each other.
With this technique, misalignment that arises during a single model's inference, and even a concrete or partial plan to act against human interests, can stay hidden and will be passed along to the next agent.
This could allow small misaligned thoughts to spread from one agent to another. If misaligned hidden thoughts arise once, even by accident, they may be preserved and even get worse over time, as agents keep passing information between them.
We have no precedent for this.
The misalignment might very well grow worse over time. Or it might be unstable. Or it might fix itself, if we are lucky. We just don't know, because as far as I am aware there are no studies on the very-long-term stability of latent thoughts like these. The paper just came out, and we don't know the edge cases yet.
Maybe the handover of latent has a tendency to monotonically increase some attribute of reasoning we were previously unaware of? It will take some time for us to discover this, and I think we should start investigating the possibility early.
One thing seems clear to me: If the performance gains are as large as the paper claims, frontier companies are almost certainly going to adopt this technique regardless of safety implications. The technique increases both accuracy and speed, while reducing token usage (and therefore cost).