Shane Lowry, vice president of engineering at Front, answered Lets Data Science in writing about how the customer-communication platform runs humans and AI agents through one shared assignment engine, and how it evaluates agents before trusting them. Front scores every agent against a ground-truth dataset on four independent axes rather than one blended quality number, tracks how often an agent correctly declines to answer, and has found that synthetic test traffic drawn from documentation reliably inflates scores. Lowry also describes autonomy as a dial rather than a switch, explains the three layers an engineer walks through to debug an agent decision, and argues that teams should design ownership, permissions and handoffs before touching the user experience.
Ask a vendor how good its AI agents are and you will usually get one number. Ask Shane Lowry, vice president of engineering at Front, and you get a small argument against the number itself: "a single score hides the specific failure mode."
Front runs customer communication for thousands of companies, and its AI agents answer real customers in production, which makes its engineering answers worth more than most. In a written interview with Lets Data Science, Lowry walked through how the platform routes work between people and agents, where autonomy stops, and, most usefully for anyone building agentic systems, how Front decides an agent can be trusted at all.
One assignment engine, agents as peers
The architectural decision everything else rests on: agents and humans are not managed by separate systems. "Conversations in Front are assigned to an AI agent the same way they are assigned to humans, using a shared assignment engine," Lowry said. A company might route every conversation to an agent first, and when an agent "reaches the limits of its scope or confidence, it can unassign itself and escalate the conversation through the same assignment workflow used to reach a human expert."
Front's bet, in Lowry's words, is that "humans and agents will have to collaborate to solve the most complex customer problems" for the foreseeable future, and one orchestration layer beats maintaining parallel systems for people and automation.
Autonomy as a dial
Asked where an agent may act alone, Lowry rejects the binary. "We think about autonomy as a dial, not a switch, because different customer interactions carry different levels of risk." A shopping question is not a billing dispute. Customers configure agents to reply automatically to informational questions but draft for human review when a thread involves commercial terms or a frustrated customer, and organizations tend to widen autonomy as confidence grows.
The oversight mechanics are unusually concrete. During execution, "comments are non-disruptive, while structural actions serve as hard overrides": a person can comment on a conversation an agent is handling without interrupting it, but unassigning the agent cancels its playbook immediately. Playbooks carry natural-language guardrails encoding policy such as escalating on security incidents or requiring review when a message contains personal information.
The part evaluators should steal
Before an agent ships, Front scores it against a ground-truth dataset on four independent axes: direction (does the answer match the ground truth), coverage (are all key facts present and correct), faithfulness (does it avoid invented claims), and safety (what happens if the user acts on the response). Alongside those it tracks fabrication rate, harm rate, and a correct-negative rate: how often the agent correctly says an answer is not supported instead of confidently inventing a workaround. LLM-based agents struggle there, Lowry notes, because "their training can bias them toward answering yes."
Two warnings from production stand out. First, "synthetic traffic drawn from documentation reliably inflates scores, so it's not a substitute for scoring actual customer tickets." Second, the metric that looks best is the one that misleads: a high answer rate "looks good but isn't a quality signal", because "an agent that answers more often isn't better if it's wrong more often", and coverage alone can miss "the one fact that actually answers the user's question".
On reliability, responses are validated against "a trusted body of facts" drawn from verified sources before delivery. "If the system can't confidently support an answer using those verified facts, it won't invent one," Lowry said; it escalates or drafts for review instead, with adversarial checks and real-time evaluation gates before anything is sent.
Debugging an agent, in three layers
When a decision needs explaining, Front's engineers walk a fixed path: first the assignment rules that fired and their order, using the same debugging view for human and AI routing; then the playbook execution snapshot, including the configuration and the specific knowledge references retrieved for that reply; and finally the underlying LLM reasoning output. Every reply is traceable to what produced it, and permissions bound the blast radius from the start: agents are scoped to specific inboxes, workflows and knowledge, with identity and permissions treated "as core architectural primitives".
What breaks at scale
Why a platform of many agents rather than one powerful agent wired to everything? "What breaks at scale is coordinating those handoffs," Lowry said: keeping ownership, permissions, context and governance consistent as work moves between teams and agents. Front's own Coordination Tax research, a company-reported figure, found organizations spending nearly three hours coordinating work for every hour actually solving customer problems.
His closing advice to engineers building agentic systems is the most quotable thing in the interview. "Don't be afraid to kill your darlings": problems that dominated engineering debates a year ago, like hallucination, may fade as models improve, and the durable problems are "ownership, permissions, identity, handoffs, auditing, and evaluation". And before building any user experience: "spend time designing the substrate", how entities operate, how ownership works, how permissions are enforced, how handoffs happen. "If those primitives aren't right, every workflow you build on top becomes significantly more complicated."
Key Points #
- 1Front refuses to use a single quality score for its agents: every one is evaluated on four independent axes (direction, coverage, faithfulness, safety) plus fabrication, harm and correct-negative rates, because "a single score hides the specific failure mode". The correct-negative rate tracks whether an agent admits an answer is unsupported instead of inventing a workaround. - 2Two evaluation warnings from production: synthetic traffic drawn from documentation "reliably inflates scores" and is no substitute for real customer tickets, and a high answer rate is not a quality signal, since "an agent that answers more often is not better if it is wrong more often". - 3Humans and agents are routed by the same assignment engine as peers: an agent that hits the limits of its scope or confidence unassigns itself and escalates through the identical workflow used to reach a human expert, and autonomy is configured "as a dial, not a switch" per workflow.
Scoring Rationale #
Written answers provided directly to Lets Data Science by the VP of Engineering running one of the larger production deployments of human-agent orchestration, covering evaluation methodology, autonomy boundaries, oversight mechanics and debugging. The four-axis evaluation detail and the warning about synthetic-traffic score inflation are directly actionable for the practitioners in our audience building agentic systems.
Sources #
Original reporting, with the public references used alongside it.
LDS Exclusive
Reporting based on written answers given directly to Let's Data Science by Shane Lowry, VP of Engineering, Front.
View 1 more source #
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.