{"slug": "magen-local-penetration-testing", "title": "Magen: Local Penetration Testing", "summary": "A developer built Magen, a local-only penetration testing copilot that captures proxy traffic, flags leaked secrets and personal data, and uses a locally run open-weight model to reason about findings and generate discovery probes. Real captured tokens are injected into probe processes via environment variables at run time so the model never sees credentials and no data leaves the machine. The developer reports the deterministic layers — capture, secret and PII detection, redaction, and statistical anomaly detection — worked reliably, while the local-model layer fell short: small models were shallow and larger ones took close to a minute per step and still misjudged endpoints and findings, a gap attributed to missing task-specific fine-tuning and over-broad initial scope.", "body_md": "*This is a submission for the [Hacktoberfest Open-Source AI Challenge Week 2: Side Hustle](https://dev.to/challenges/hacktoberfest-week2-2026-10-12)*\n\nMagen is a local-only security testing copilot. It captures traffic while you have the proxy running, flags leaked secrets and personal data, and uses a local open-weight model to reason about what it found and to write and run discovery probes. The one thing that defines it: every part runs on your machine. Nothing sensitive leaves the laptop.\n\nThe problem it solves is specific. When you security test a real application you are handling live session tokens, .env files, and other people's personal data. You cannot paste any of that into a hosted LLM without creating a new liability. A model running locally removes that constraint, so the tool can reason over the real data instead of a redacted shadow of it.\n\nWho would pay for it: freelance security testers, small teams, and bug bounty hunters who want an AI assistant but cannot ship client data to a third party.\n\nWhat it does today:\n\nThe open pieces are the project, not an accessory to it:\n\nThe design choice I am happiest with is how the agent handles secrets. When the model writes a probe to test an endpoint, Magen injects the real captured token into the probe's process through an environment variable at run time. The model never sees the token and never writes it into code, yet the probe still authenticates for real. The model reasons, the machine holds the secret, and nothing leaves the device.\n\nThis is the whole reason the project exists. Three things a closed API could not give me:\n\nRunning an open model locally was not a nice-to-have here. It was the only honest way to build something that touches credentials. While a hosted open-weight model could have done work here, I felt it went against the principals of what I was trying to accomplish, and that meant making external requests.\n\nThis project ended up being more of a learning experience than a finished product, and I think that is worth noting.\n\nThe deterministic parts worked well: capture, secret and PII detection, redaction, and the statistics-based anomaly detection. Those are reliable, genuinely useful on their own, and they need no model.\n\nThe local-model layer is where I fell short. The models that run comfortably on my hardware were not quite strong enough for the agentic security reasoning I was asking for. The small model is fast but shallow. The larger one reasons better but takes close to a minute per step and was still inconsistent; it would reach for libraries that were not available, pick weak endpoints to probe, or misjudge whether a response was actually a finding. That is not the model being bad. It is a mismatch between a general-purpose local model with no task-specific tuning and the fairly precise job I wanted it to do.\n\nTwo things would have closed most of that gap, and I had time for neither: fine-tuning a model for this narrow task, and simply iterating longer on the prompts and the evidence format the model has to produce.\n\nThe bigger lesson was about scope. I started too broad. Under time pressure I kept adding features: a service manager, a one-click demo, a managed test browser, multiple probe goals. Each one widened the surface area, and each one introduced bugs I then had to chase instead of improving the core. A tighter starting scope, something like \"capture traffic, scan it for secrets locally, and produce a redacted report,\" would have been a smaller and far more solid product, with the local model doing one job it could actually do well. Improper scoping at the very start was probably the single biggest thing working against me.\n\nSo I do not read this as a failure. The privacy thesis holds up, the deterministic tooling is good, and I now know exactly where a local-first AI tool earns its keep and where it still needs either a stronger model or real tuning time. That is a useful thing to carry into the next build.", "url": "https://wpnews.pro/news/magen-local-penetration-testing", "canonical_source": "https://dev.to/gavin_rose/magen-local-penetration-testing-c9h", "published_at": "2026-10-19 00:44:06+00:00", "updated_at": "2026-10-19 00:48:53.374739+00:00", "lang": "en", "topics": ["ai-tools", "ai-agents", "ai-safety", "artificial-intelligence"], "entities": ["Magen", "Hacktoberfest"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/magen-local-penetration-testing", "markdown": "https://wpnews.pro/news/magen-local-penetration-testing.md", "text": "https://wpnews.pro/news/magen-local-penetration-testing.txt", "jsonld": "https://wpnews.pro/news/magen-local-penetration-testing.jsonld"}}