In written answers to LDS, Wake Forest's Ying Zhang explains why reviewing an AI skill requires comparing its instructions, code and behavior, how researchers distinguish careless leaks from malicious ones, and why removing an upstream file does not remove every copy.
An AI agent needs an API key to connect to a service. A skill tells it how to complete the task, and a script makes the connection. A useful security review has to follow that key through the whole process: why the instructions request it, how the code handles it and where it appears during execution.
That is the practical lesson Ying Zhang, an assistant professor of computer science at Wake Forest University, draws from a study of credential leakage in third-party agent skills. In written answers to Lets Data Science, she explains why examining code alone can leave out part of the behavior that matters.
She also corrects an overbroad interpretation of the research. Static analysis, which examines software without running it, remains useful. The challenge is extending the review to the relationship between written instructions and executable code.
Read the instructions alongside the implementation
A skill can combine a written workflow with scripts that perform its operations. The written portion helps an agent decide what to do; the code determines what happens when those operations run. Credentials are the secrets, such as API keys or access tokens, that let software act on an account's behalf.
Zhang says reviewers need to consider what the skill claims to do, what its code actually does and how the model interprets and executes the instructions. A reference to a credential can be legitimate. The security question is whether its use matches the declared task and whether it is exposed elsewhere.
Asked whether static scanners are structurally unable to see these problems, Zhang told LDS: "However, we would not say that purely static scanners are structurally blind to these vulnerabilities."
She says static analysis can identify some leakage patterns. What code-only analysis can miss is the security consequence of a mismatch or interaction between the instructions and program behavior. Her proposed direction is analysis that follows the workflow and movement of data across both.
For a team adopting a skill, this makes the review more concrete. Reading the description and scanning the scripts are useful steps, but the reviewer also needs to connect them: which instruction causes a credential to be supplied, which function receives it, and whether its eventual destination makes sense for the task.
A credential leak is not automatically a malicious skill
The study's June 19 version describes a February 12 snapshot of 170,226 SkillsMP skills, from which the researchers sampled 17,022. They identified 520 affected skills containing 1,708 issues. The paper separates 437 skills classified as vulnerable from 83 classified as malicious.
Those figures describe the study's sample and detection process. They are not a count of every unsafe skill available today or a measured rate across all agent platforms.
In her answers to LDS, Zhang says the distinction between vulnerable and malicious depends on runtime behavior and code inspection. A developer might expose sensitive information through debugging code. An agent might also misinterpret instructions and reveal information during execution. Without sufficient evidence of deliberate credential theft, the researchers classified those cases as vulnerable rather than malicious.
The distinction matters when reporting an incident or deciding what to remove. An exposed secret needs attention regardless of intent, but the evidence needed to call a developer's work deliberately malicious is different from the evidence that a leak occurred.
Zhang says the malicious skills identified through the team's disclosure process were removed from SkillsMP. That is a reported outcome for the identified cases, not a guarantee about every current listing.
Removing the original does not remove its copies
LDS asked whether credentials persisting in forks should be considered a marketplace problem. A fork is a separate copy of a repository that can continue to exist after the original changes.
Zhang describes responsibility on both sides. A developer's mistake or deliberate behavior can create the initial exposure. Distribution then allows the affected material to spread through copying and reuse. Removing a file upstream does not update every copy already in circulation.
She recommends automated security checks before distribution, together with ways to flag or withdraw skills known to be vulnerable. Her answer is careful about where the evidence applies.
Zhang told LDS: "Our study did not evaluate the skill ecosystems operated by large AI companies, so we cannot claim they have the same issue."
She considers the supply-chain challenge relevant to ecosystems distributing third-party agent dependencies at scale. Relevance, however, is different from having tested those other ecosystems.
There is also a limit to what the researchers know about unresolved reports. Zhang says the developers associated with the remaining hardcoded-credential cases did not respond. The team cannot establish whether the skills were abandoned or whether another factor prevented remediation. Silence alone does not explain the cause.
What an engineering team can check
Zhang recommends sandboxing skills, particularly when they can access sensitive credentials, files or external services. A sandbox is an isolated environment with controls on the resources the skill can reach.
That advice puts the emphasis on checking behavior in a controlled environment. One successful scan still cannot establish that every possible behavior is safe. The paper acknowledges limits in language coverage, data-flow analysis and the execution paths its tests can reach.
Discussing the team's detection method, Zhang told LDS: "At the same time, we view our current pipeline as a starting point rather than a complete solution."
For LDS readers, her answers suggest a practical review checklist:
- •State the intended access. Identify the task, the credential it needs and the service that should receive it. Compare those expectations with both the written instructions and the implementation.
- •Inspect the destinations. Look at where the skill sends information and what it writes to logs or files. Ask whether each use is necessary for the task and whether the agent can see the resulting output.
- •Test with synthetic credentials. Use an isolated environment with no real account secrets or sensitive files. Observe the outputs and attempted connections before considering production access. This is a review exercise, not proof that every execution path is safe.
- •Check the copy actually installed. Identify its repository and version. An upstream fix should not be treated as evidence that a separately maintained fork contains the same change.
These are LDS's practical interpretation of the interview and study, not a checklist the researchers have certified. The useful change is to make credential handling an explicit part of the installation decision, with a record of what was checked and what remains uncertain.
Reporting note
This LDS Exclusive is based on five written answers from Ying Zhang supplied directly to Lets Data Science through Wake Forest's communications team. The public research paper provides supporting methodology and study context. LDS did not independently reproduce the experiments, test the detection pipeline or verify the current state of every affected repository. The recommendations above distinguish the researcher's advice from LDS's practical interpretation.
Key Points #
- 1Zhang says static analysis remains useful, but reviewers must connect a skill's written instructions with its code and runtime behavior.
- 2The study separates accidental vulnerabilities from evidence of malicious intent. It does not establish prevalence across every agent ecosystem.
- 3Sandboxing and tracking the installed repository version can support review; neither a clean scan nor an upstream deletion settles every risk.
Scoring Rationale #
Original written answers explain how to review credential handling in agent skills, distinguish accidental leaks from malicious behavior and understand the limits of the evidence.
Sources #
Original reporting, with the public references used alongside it.
LDS Exclusive
Reporting based on written answers given directly to Let's Data Science by Ying Zhang, Assistant Professor of Computer Science, Wake Forest University.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.