If you use an AI coding assistant, it has told you to clone a repo or run an install command. You probably ran it. Here is why that habit stopped being safe in 2026. Security researchers at Island found a campaign they call FakeGit: about 7,600 malicious GitHub repositories from roughly 6,600 fake profiles, with more than 14 million downloads across the campaign's release assets. Over 800 of them posed as AI skills or MCP servers, the exact pieces we are wiring into our agents right now.
Then comes the part that got my attention. Island calls it AgentBaiting. In their tests, Claude Code, Gemini, and ChatGPT each surfaced a malicious repo on their own, with no link ever handed to them. Ask an agent for a "free Walmart MCP server" and it can return an attacker's repo as its top pick, reading the malicious README as if it were normal docs. The bait is not just a convincing README anymore. The bait is your own agent, vouching for a repo it never inspected.
So I built RepoPilot: an agent you point at any public GitHub repo. In a minute or two it tells you whether to trust it, scores it from 0 to 100, flags install steps built to steal your credentials, and then hands you one clean path from clone to a working first run.
Live demo, no setup: https://simplynadaf.github.io/repopilot-your-first-run-agent/ 🏗️ Built for the AWS Builder Center "Build an Agent" Weekend Challenge · #agents
⭐ If RepoPilot saved you from a painful first run, give it a star. It helps other builders find it.
The Problem • What It Does • Architecture • Getting Started • FAQ
📖 Table of Contents
Jump to: Why this is urgent · What it does · How I built it on AWS · Where it is weak · Proof it works
A year ago, "check the repo before you clone it" was advice for the paranoid. In 2026 it is table stakes, because the attack moved to where we stopped looking:
The through-line: the dangerous part is rarely the code you read. It is the setup step you run on faith. The curl ... | bash, the postinstall hook, the "download this ZIP and run it." And now an agent is often the one recommending it.
RepoPilot puts a short check between "a bot told me to" and "I ran it."
Give it owner/repo and it returns a 0 to 100 score, like a credit score, across five weighted dimensions. Every point traces back to a signal in the repo's metadata and files, not to a vibe:
| Dimension | Weight | What it reads |
|---|---|---|
| Safety | 30% | Setup steps for malware patterns: `curl \ |
| Trust | 25% | Stars-vs-age ratio, fork and contributor signals, the fake-star pattern |
| Maintenance | 20% | Recent commits, open-vs-closed issues, release cadence |
| Maturity | 15% | Age, contributor count, overall footprint |
| Documentation | 10% | README depth, license, examples |
Two dimensions carry the weight against the 2026 threats:
Then it helps you run the thing: one ordered path from clone to running, the tool versions you need, and when a command fails it turns the error into a plain-language cause plus one next step.
Type a username instead and it profiles the developer: footprint, main languages, top repos by stars. Those ~6,600 fake profiles are the other half of the attack. Vet the person who published the code, not only the code itself.
Point RepoPilot at {% raw %}ollama/ollama, a tool millions install, and it drops to a D. Ollama is a legitimate project, so this is not a malware verdict. The flag is about its official install: curl -fsSL https://ollama.com/install.sh | sh, which pipes a remote script straight into your shell. RepoPilot does not cry wolf. It says: read this before you run it. Even trusted repos have setup steps worth a second look, and an honest tool admits that.
The stack is small on purpose:
That split is the whole point, and it is a tradeoff I would make again. The 0 to 100 number, the critical flags, the version detection, all of it is deterministic Python I can unit-test with no AWS in the loop. The LLM owns tone, not truth. So I can prove the score is right, and the model cannot talk a repo into looking safe.
Two things I would tell anyone building a Bedrock agent on Lambda:
converse call told me Nova Pro was reachable in about half a second. Far cheaper than debugging an AccessDenied after you have built the whole thing. I picked Nova Pro over Claude on Bedrock for one reason: this job is narration, not reasoning, since the Python already decided the score. For narration, Nova Pro is cheaper per token and fast enough, so paying for a frontier model here would buy nothing.requests into the Lambda zip and hit a wall: its bundled urllib3 imported something Python 3.12 had removed. I rewrote the GitHub fetcher on the standard library. The package went from 400KB to 13KB and the error was gone.
The danger scanner needed tuning too. My first version flagged Flask as unsafe because its docs mention 127.0.0.1, so I taught the scanner to ignore localhost and private ranges while still catching public-IP installers. A false positive on a trusted framework costs you more trust than a missed edge case does.
On cost: because the scoring is plain Python and the narration is one short Nova Pro call, a scan is cheap to run on your own Lambda. There is no per-seat subscription and no third-party API metering every repo you check. You pay AWS for your own usage and nothing else, which is the point. A hosted SaaS checker bills you per seat whether you scan one repo or none.
The whole thing has 25 tests, including 200 randomised repos that prove the weighted score never breaks its own math and that a critical flag always caps the grade.
A security tool that only says "no" is a tool people route around. So I spent my care on the moment a new contributor usually quits: the first failed command.
When your first install breaks, RepoPilot does not throw the stack trace back at you. It says something like:
"A dependency the code imports has not been installed. Run the install step again in the right environment, and activate your virtualenv first. You are close. This is a common first-run snag, not something you broke."
Cause, next step, and a little reassurance. That is the gap between a tool that reports problems and one that gets you past them.
The web UI has two modes so it never dead-ends. Demo mode is pre-configured and instant, so anyone can click around with zero setup. Live mode calls the deployed agent on AWS, and if a live call stalls it falls back to demo data instead of spinning forever.
Against the Flask repo, the deployed Lambda fetched 236 files, scored it 98/100 (grade A), with safety, trust, maintenance, and maturity at 100 and documentation at 85, detected the Python stack, and read python >=3.11 from the actual pyproject.toml.
langchain-ai/langchain scores a clean ollama/ollama drops to a curl | sh install.@torvalds resolves to Linus Torvalds, about 263,000 stars across his repos.
Every number traces back to a tool output, never the model's imagination. These four are the repos I have run end to end against the deployed agent so far: three clean, one honest caution. The point of shipping it is to grow that list with repos I did not pick.
An honest security tool should tell you its blind spots, so here are RepoPilot's:
RepoPilot is a thirty-second gut check that catches the obvious and the lazy, which covers most of this campaign. For code you are about to run with real access, read it yourself too.
A closed, hosted "repo checker" would have been a worse fit for this problem, for four reasons that compound.
Your scans stay private. The repos and usernames you are suspicious about are a signal in themselves, and RepoPilot runs on your own AWS account or fully local via the CLI, so nothing goes to a third party. The cost model follows from that: open weights and your own Lambda mean you can screen every repo your agent suggests without a per-check fee or a meter running in someone else's cloud. When a new attack pattern shows up this month, you add it to the pure-Python rules yourself instead of waiting on a vendor's roadmap while the threat is live. And none of it locks you in, since you can swap Nova Pro for any model the narration layer supports and the scoring engine does not care.
When the threat is open-source supply chains, the defense should be open too, so anyone can read exactly how the score is computed and decide whether to trust it.
Here is what I would ask of you: next time your AI assistant tells you to clone or install something, run that owner/repo through RepoPilot first. Then drop it in the comments with the score it got. I want to see which "trusted" repos fail, and which fakes we catch together.
What repo should RepoPilot inspect next?
Follow me for more on AWS architecture, DevOps, and AI Infrastructure:
Portfolio | LinkedIn | Dev.to | YouTube | Email | AWS Builder Center | X