PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?

A new evaluation framework called PACT is proposed to test whether enterprise-grade LLM agents comply with rules specified in their system context when deployed in sensitive domains such as hiring, healthcare, and finance, where compliance is a first-order legal concern. The proposal responds to the absence of any existing evaluation framework for measuring that compliance under pressure as corporate AI adoption grows.

As corporate AI adoption continues to grow, enterprise-grade LLM agents are being deployed into sensitive contexts such as hiring, healthcare, and finance. In these contexts, compliance with rules specified in an agent's system context is a first-order legal concern. Currently, no evaluation framewo