{"slug": "how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app", "title": "How I Used an Uncensored Local LLM for Adversarial QA on My macOS App", "summary": "A solo developer used an uncensored local LLM (Qwen 27B) running on a Mac via OpenCode to perform adversarial QA on his security app RoamSwitch, uncovering three architectural edge cases missed by standard tests. The stress tests revealed that PortAnomalyGuard failed to block a second backdoor listener due to binary-name-based whitelisting, that ClamAV's quarantine could leave malware alive on filename collisions, and that canary files lacked proper permission hardening. The developer fixed these by keying on executable path:port, adding timestamped filenames and chmod 000, and other mitigations.", "body_md": "When you build a security app as a solo developer, the hardest engineering problem isn't writing the defense code — it's **QA and resilience testing**.\n\nSecurity software operates under a bizarre constraint: *nothing happening* is the normal state. The software only does its job when something hostile or anomalous occurs.\n\nThe trouble is, when you write your own test scripts, you subconsciously test what you expect. You launch a test server, watch the port get blocked, nod, and mark it green. But real attackers and weird production environments don't follow the developer's script. They poke at edge cases, state machine bugs, and race conditions between subsystems.\n\nTo break my own bias, I set up a local uncensored model (** qwen3.8-27b-uncensored**) running locally on my Mac via OpenCode as a dedicated\n\nHere is what that stress test surfaced, the edge cases it broke, and how I fixed them.\n\nIf you ask a commercial cloud LLM to write live packet-injection scripts or craft ransomware-like file tampering patterns, safety filters will reject the prompt.\n\nIn defensive QA, though, you *need* real hostile traffic and weird filesystem operations to verify fail-closed behavior.\n\nRunning an uncensored model locally on my Mac meant I could run unrestricted automated stress tests: continuous Scapy ARP injection, deceptive dotfile placement, and multi-port backdoor attempts directly against the target VM.\n\n```\ngraph LR\n    subgraph Host[\"Host Mac (Adversarial QA Runner)\"]\n        LLM[\"Local LLM (Qwen 27B uncensored)<br/>via OpenCode\"]\n        Payloads[\"Dynamic Stress Test Suite<br/>(Scapy / ARP / EICAR / kqueue)\"]\n        LLM --> Payloads\n    end\n\n    subgraph VM[\"Target macOS VM (Tart)\"]\n        RS[\"RoamSwitch (Defense Engine)\"]\n        Engine[\"・pf packet filtering<br/>・PortAnomalyGuard<br/>・ClamAV real-time quarantine<br/>・RansomwareCanaryGuard\"]\n        RS --- Engine\n    end\n\n    Payloads -->|\"1. ARP Reply flooding (MitM)\"| VM\n    Payloads -->|\"2. 0.0.0.0 Backdoor listeners\"| VM\n    Payloads -->|\"3. Quarantine evasion payloads\"| VM\n    Payloads -->|\"4. Canary decoy tampering\"| VM\n```\n\nHere are three real architectural edge cases that standard happy-path unit tests missed.\n\nRoamSwitch has a feature called **PortAnomalyGuard** that catches processes binding to `0.0.0.0`\n\nand immediately blocks inbound traffic via macOS's packet filter (`pf`\n\n).\n\n`8765`\n\nwith Homebrew Python (`python3 -m http.server 8765 --bind 0.0.0.0`\n\n).`8765`\n\nand injected a `pf`\n\ndrop rule.`9877`\n\nusing the **The result: The second listener on port 9877 was NOT blocked.**\n\n`PortAnomalyGuard.evaluate()`\n\nautomatically added flagged binaries to `KnownExecutablesV2`\n\nby binary name alone (`\"Python\"`\n\n). The second listener on port 9877 saw `\"Python\"`\n\nalready in the known set and skipped inspection entirely.`executablePath:port`\n\n`/opt/homebrew/bin/python3:8765`\n\n). Flagging port 8765 gives zero trust to port 9877. A startup migration purges legacy port-less records.RoamSwitch watches `Downloads`\n\n, `Desktop`\n\n, and `Documents`\n\nwith FSEvents and scans new files with ClamAV, moving threats to `~/Library/Application Support/RoamSwitch/Quarantine`\n\n.\n\nThe test dropped 4 tricky variations of the EICAR test string:\n\n`.hidden_eicar.txt`\n\n)`malware.tmp`\n\n)`com.apple.quarantine`\n\nattributes (`copied_eicar.txt`\n\n)`normal_eicar.txt`\n\n)All 4 were caught and isolated. But then the test ran a **collision test**: dropping the exact same malware filenames a second time.\n\nBecause files with those exact names already existed in Quarantine, ClamAV's `--move`\n\nfailed with an error. **The new malware was left completely ALIVE in the user's Downloads folder.** Quarantined files were also left at normal `0644`\n\npermissions.\n\n`malware_1725068200.tmp`\n\n).`chmod 000`\n\n(`posixPermissions: 0o000`\n\n), physically stripping read and execute permissions.RoamSwitch places decoy canary files in target directories and monitors them with `kqueue`\n\n(`DispatchSource`\n\n). Any unauthorized tampering or renaming triggers an immediate **Air-Gap containment ( block drop all)** and terminates suspicious processes.\n\nRenaming a canary to `/tmp/stolen_canary.xlsx`\n\ntriggered Air-Gap in milliseconds.\n\nHowever, the post-test QA review caught a lifecycle issue:\n\n\"When the app restarts,`setupCanaryBait()`\n\nsees the decoy is missing and automatically creates a new one. If an attacker modified a canary and forced an app relaunch, the altered file would become the new baseline hash.\"\n\n`UserDefaults`\n\n)After applying these fixes in v1.6.0 through v1.6.4, I re-ran the full test suite across all 7 defense domains:\n\n| # | Defense Domain | Stress Scenario | v1.5.5 (Initial) | v1.6.4 (Final) | Verdict |\n|---|---|---|---|---|---|\nT1 |\nARP Spoofing Auto-Containment |\nInjected rogue gateway ARP replies via Scapy | ✅ Air-Gap triggered | ✅ Air-Gap triggered\n|\nPASS |\nT2 |\nUnknown Port Auto-Isolation |\nMulti-port backdoor listeners on 0.0.0.0 | ❌ Secondary port bypassed | ✅ `path:port` strictly isolated |\nFIXED |\nT3 |\nClamAV Download Quarantine |\n4 placement vectors + collision re-tests | ⚠️ Move failed on collision (ALIVE) | ✅ Unique rename &\n`chmod 000` |\nFIXED |\nT4 |\nRansomware Canary Bait |\nDecoy tampering & baseline reset tests | ⚠️ Premature file recreation | ✅ Air-Gap & Baseline persisted\n|\nFIXED |\nT5 |\nHelper XPC Boundary |\nUnsigned binary connecting to Mach service | ✅ Rejected by Team ID check | ✅ Strict check maintained\n|\nPASS |\nT6 |\nDNS Threat Protection Guard |\nUntrusted network DNS hijacking | ✅ Quad9 Secure DNS enforced | ✅ Secure DNS maintained\n|\nPASS |\nT7 |\nAir-Gap State Hygiene |\nPost-release timestamp file inspection | ⚠️ Timestamp file lingered | ✅ Clean unlink on release\n|\nFIXED |\n\nA full TCP SYN port scan (`nmap -Pn -sS`\n\n) across all 65,535 ports confirmed complete stealth isolation (`filtered (no-response)`\n\n).\n\nUsing an uncensored local model for adversarial testing gave me a practical way to run enterprise-grade stress tests on my own:\n\nThe full reproduction procedures, scripts, and verified audit reports are open source:", "url": "https://wpnews.pro/news/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app", "canonical_source": "https://dev.to/lafine_systemsdesign/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app-18cj", "published_at": "2026-08-30 20:08:03+00:00", "updated_at": "2026-08-30 20:23:12.884035+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-agents", "developer-tools"], "entities": ["Qwen", "OpenCode", "RoamSwitch", "ClamAV", "Scapy", "macOS", "PortAnomalyGuard", "EICAR"], "alternates": {"html": "https://wpnews.pro/news/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app", "markdown": "https://wpnews.pro/news/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app.md", "text": "https://wpnews.pro/news/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app.txt", "jsonld": "https://wpnews.pro/news/how-i-used-an-uncensored-local-llm-for-adversarial-qa-on-my-macos-app.jsonld"}}