When you build a security app as a solo developer, the hardest engineering problem isn't writing the defense code — it's QA and resilience testing.
Security software operates under a bizarre constraint: nothing happening is the normal state. The software only does its job when something hostile or anomalous occurs.
The trouble is, when you write your own test scripts, you subconsciously test what you expect. You launch a test server, watch the port get blocked, nod, and mark it green. But real attackers and weird production environments don't follow the developer's script. They poke at edge cases, state machine bugs, and race conditions between subsystems.
To break my own bias, I set up a local uncensored model (** qwen3.8-27b-uncensored**) running locally on my Mac via OpenCode as a dedicated
Here is what that stress test surfaced, the edge cases it broke, and how I fixed them.
If you ask a commercial cloud LLM to write live packet-injection scripts or craft ransomware-like file tampering patterns, safety filters will reject the prompt.
In defensive QA, though, you need real hostile traffic and weird filesystem operations to verify fail-closed behavior.
Running an uncensored model locally on my Mac meant I could run unrestricted automated stress tests: continuous Scapy ARP injection, deceptive dotfile placement, and multi-port backdoor attempts directly against the target VM.
graph LR
subgraph Host["Host Mac (Adversarial QA Runner)"]
LLM["Local LLM (Qwen 27B uncensored)<br/>via OpenCode"]
Payloads["Dynamic Stress Test Suite<br/>(Scapy / ARP / EICAR / kqueue)"]
LLM --> Payloads
end
subgraph VM["Target macOS VM (Tart)"]
RS["RoamSwitch (Defense Engine)"]
Engine["・pf packet filtering<br/>・PortAnomalyGuard<br/>・ClamAV real-time quarantine<br/>・RansomwareCanaryGuard"]
RS --- Engine
end
Payloads -->|"1. ARP Reply flooding (MitM)"| VM
Payloads -->|"2. 0.0.0.0 Backdoor listeners"| VM
Payloads -->|"3. Quarantine evasion payloads"| VM
Payloads -->|"4. Canary decoy tampering"| VM
Here are three real architectural edge cases that standard happy-path unit tests missed.
RoamSwitch has a feature called PortAnomalyGuard that catches processes binding to 0.0.0.0
and immediately blocks inbound traffic via macOS's packet filter (pf
).
8765
with Homebrew Python (python3 -m http.server 8765 --bind 0.0.0.0
).8765
and injected a pf
drop rule.9877
using the The result: The second listener on port 9877 was NOT blocked.
PortAnomalyGuard.evaluate()
automatically added flagged binaries to KnownExecutablesV2
by binary name alone ("Python"
). The second listener on port 9877 saw "Python"
already in the known set and skipped inspection entirely.executablePath:port
/opt/homebrew/bin/python3:8765
). Flagging port 8765 gives zero trust to port 9877. A startup migration purges legacy port-less records.RoamSwitch watches Downloads
, Desktop
, and Documents
with FSEvents and scans new files with ClamAV, moving threats to ~/Library/Application Support/RoamSwitch/Quarantine
.
The test dropped 4 tricky variations of the EICAR test string:
.hidden_eicar.txt
)malware.tmp
)com.apple.quarantine
attributes (copied_eicar.txt
)normal_eicar.txt
)All 4 were caught and isolated. But then the test ran a collision test: dropping the exact same malware filenames a second time.
Because files with those exact names already existed in Quarantine, ClamAV's --move
failed with an error. The new malware was left completely ALIVE in the user's Downloads folder. Quarantined files were also left at normal 0644
permissions.
malware_1725068200.tmp
).chmod 000
(posixPermissions: 0o000
), physically stripping read and execute permissions.RoamSwitch places decoy canary files in target directories and monitors them with kqueue
(DispatchSource
). Any unauthorized tampering or renaming triggers an immediate Air-Gap containment ( block drop all) and terminates suspicious processes.
Renaming a canary to /tmp/stolen_canary.xlsx
triggered Air-Gap in milliseconds.
However, the post-test QA review caught a lifecycle issue:
"When the app restarts,setupCanaryBait()
sees the decoy is missing and automatically creates a new one. If an attacker modified a canary and forced an app relaunch, the altered file would become the new baseline hash."
UserDefaults
)After applying these fixes in v1.6.0 through v1.6.4, I re-ran the full test suite across all 7 defense domains:
| # | Defense Domain | Stress Scenario | v1.5.5 (Initial) | v1.6.4 (Final) | Verdict |
|---|---|---|---|---|---|
| T1 | |||||
| ARP Spoofing Auto-Containment | |||||
| Injected rogue gateway ARP replies via Scapy | ✅ Air-Gap triggered | ✅ Air-Gap triggered | |||
| PASS | |||||
| T2 | |||||
| Unknown Port Auto-Isolation | |||||
| Multi-port backdoor listeners on 0.0.0.0 | ❌ Secondary port bypassed | ✅ path:port strictly isolated |
|||
| FIXED | |||||
| T3 | |||||
| ClamAV Download Quarantine | |||||
| 4 placement vectors + collision re-tests | ⚠️ Move failed on collision (ALIVE) | ✅ Unique rename & | |||
chmod 000 |
|||||
| FIXED | |||||
| T4 | |||||
| Ransomware Canary Bait | |||||
| Decoy tampering & baseline reset tests | ⚠️ Premature file recreation | ✅ Air-Gap & Baseline persisted | |||
| FIXED | |||||
| T5 | |||||
| Helper XPC Boundary | |||||
| Unsigned binary connecting to Mach service | ✅ Rejected by Team ID check | ✅ Strict check maintained | |||
| PASS | |||||
| T6 | |||||
| DNS Threat Protection Guard | |||||
| Untrusted network DNS hijacking | ✅ Quad9 Secure DNS enforced | ✅ Secure DNS maintained | |||
| PASS | |||||
| T7 | |||||
| Air-Gap State Hygiene | |||||
| Post-release timestamp file inspection | ⚠️ Timestamp file lingered | ✅ Clean unlink on release | |||
| FIXED |
A full TCP SYN port scan (nmap -Pn -sS
) across all 65,535 ports confirmed complete stealth isolation (filtered (no-response)
).
Using an uncensored local model for adversarial testing gave me a practical way to run enterprise-grade stress tests on my own:
The full reproduction procedures, scripts, and verified audit reports are open source: