In written answers to LDS, Fleet CTO Luke Heath explains how Project Glasswing tests use isolated cloud deployments, synthetic data and engineer approval. Customer endpoint data is excluded, Fleet says. The interview separates those controls from the performance results still to come.
Fleet is testing what a frontier AI model can find in its software inside temporary cloud environments that contain synthetic data. In written answers to Lets Data Science, Luke Heath, CTO of Fleet Device Management, described where those tests run, what the model can do and why an engineer still has to reproduce a proposed finding.
The details give Fleet's participation in Project Glasswing a more concrete shape. The company announced on September 15 that it had joined Anthropic's defensive security program. Its work is focused on its own software and device-management protocols. Heath said the announcement does not introduce a new Mythos-powered feature for customers.
For teams assessing AI security tools, the useful question is what sits around the model: the data it receives, the authority it has and the evidence required before anyone acts on its output.
A temporary copy, with customer data left out
Heath described three inputs to the research: Fleet's public source code; security context, including its threat model and previous advisories; and dedicated test environments. When the model needs to interact with a running system, the team deploys a temporary copy of Fleet in a separate cloud account containing synthetic data.
Heath told LDS: "Customer endpoint data is not used in this work."
He added that no data from customer environments is shared with the model or Anthropic. That is a statement about this research workflow. It should not be generalized into a claim about every AI integration a Fleet customer might separately use.
Within the test deployment, the model can execute test steps and probe the system. Heath said credentials are limited to that environment, a Fleet security engineer supervises the session, and anything extending beyond it requires that engineer's approval. The environment is removed afterward.
Access to the model is restricted to named members of Fleet's security team, individually authenticated with phishing-resistant multifactor authentication, according to Heath. He described these safeguards as already in place, while broader coverage across the release process remains planned.
This distinction matters when evaluating an agent. A convincing analysis is only one part of the system; the permissions around it determine what an incorrect action could affect. Fleet's account gives readers specific controls to ask about without establishing that LDS has independently audited their implementation.
A proposed finding has to survive human reproduction
In the exercise Heath described, the model approaches the temporary deployment from outside, without application credentials, and proposes possible attack paths. The human approval step determines which tests it may attempt.
"The approval step sits in the middle: a security engineer decides what the model may try, then reproduces each candidate finding by hand before anyone treats it as real."
That process separates a plausible model response from a vulnerability the team has actually verified. Heath said confirmed findings enter Fleet's existing remediation and disclosure process. Sometimes the model identifies a problem the team already knows about. Such a result can help check a capability, but it is not a newly discovered vulnerability.
Fleet's public engineering handbook describes an engineer's initial review of security reports and a process for staging fixes while limiting disclosure of unresolved vulnerabilities. Its security handbook describes severity-based remediation and exceptions. These documents provide background on the process; they are not published results from the new Glasswing work.
Heath did not disclose an individual Glasswing finding, a reproducible test result or measured performance improvement. He said detailed findings would follow after fixes have shipped and the material has been reviewed for safe disclosure.
Research access is different from a customer product
Anthropic describes Project Glasswing as a defensive security initiative using Claude Mythos Preview. Fleet's own announcement and answers describe how it intends to apply that access to software and infrastructure it is authorized to test.
Heath separately described endpoint-management functions Fleet already provides, including checking device state, deploying changes and reporting patch status. He said those existing functions do not depend on Glasswing or Mythos Preview.
That distinction keeps the announcement in proportion. Joining a research program does not establish that customers can use its model, that a new autonomous patching service has shipped, or that a measured improvement applies to their devices.
The result still to measure
LDS asked how Fleet would determine whether AI improves on its existing security work. Heath said the team plans to compare AI-assisted work with its current engineering and security processes.
The measures he identified include valid issues missed by other tools or reviews, incorrect findings, human review effort, the accuracy of suggested fixes and the time needed to reach a validated repair. Fleet has not supplied results across enough Glasswing cases to support broad performance claims.
For a practitioner, these measures belong together. Finding more candidates is useful only if the team can establish which are real and resolve them. Faster model output can still leave engineers with more verification work. Fleet's answers set out what it plans to assess; they do not yet show the outcome of that assessment. The patching survey cited around the announcement answers a separate question. Fleet's July report, The road to AI in IT, describes May fieldwork among senior IT leaders at organizations with at least 2,000 employees. It predates the September announcement and is not an evaluation of Glasswing.
Fleet supplied LDS with the exact patching question and its 527 responses. Of those respondents, 112, or 21.3% rounded, selected less than 24 hours to deploy a critical security patch. These are respondents' estimates, not independently recorded installation times. The supplied question does not specify the timing boundaries, and two longer-duration answer labels overlap. Those limitations make it unsuitable as evidence that the new research has already improved patching.
Recovery must be checked on the device
One of Heath's practical points applies to ordinary automation as well as AI. A successful command does not establish that the intended change reached a device.
He explained that Fleet can query endpoint state after deployment to check the resulting software version, configuration or other relevant condition. Reverting the desired configuration after a failed change also needs a subsequent check that the device has returned to the expected state.
Heath recommended that teams without access to Mythos Preview start by putting device policies and configurations into version control, requiring human review and testing changes on a small group before a wider rollout.
The same discipline can guide an evaluation of an AI tool: define what it may access, give proposed findings an independent check, and decide in advance what evidence will establish success or recovery. Fleet has provided a concrete account of those boundaries. The next reporting milestone is evidence of what its research finds and how much work it takes to turn a finding into a verified fix.
Reporting note
Based on written answers from Luke Heath supplied directly to Lets Data Science on September 16, Fleet's September 15 announcement, its July survey report and public Fleet and Anthropic documentation. LDS checked the survey calculation and quoted answers but did not run the security tests or audit the controls. Descriptions of the research are attributed to Fleet. Portrait of Luke Heath supplied by Fleet for this article; used with his written permission.
Key Points #
- 1Fleet says its Glasswing research uses temporary isolated deployments with synthetic data and excludes customer endpoint data.
- 2An engineer supervises the tests and reproduces candidate findings before they enter the remediation process.
- 3No Fleet-specific Glasswing benchmark or new Mythos-powered customer feature has been established by these answers.
Scoring Rationale #
Original written answers to LDS from Fleet CTO Luke Heath explain isolated AI security tests, synthetic data, engineer approval, finding reproduction and endpoint verification. The reporting distinguishes operational controls from results still to be measured and from existing customer capabilities.
Sources #
Original reporting, with the public references used alongside it.
LDS Exclusive
Reporting based on written answers given directly to Let's Data Science by Luke Heath, CTO, Fleet Device Management.
View 1 more source #
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.