My app's landing page says 'Zero Telemetry'. So I wiretapped my own app to prove it.

A solo developer of the macOS network-security app RoamSwitch wiretapped his own app to prove its 'Zero Telemetry' claim, using tcpdump and free tools to capture and analyze outbound traffic. The procedure, documented in a blog post, verifies that only four whitelisted destinations are contacted and can be re-run for every release to catch regressions.

A Japanese version of this is on Zenn. I build and sell a macOS network-security app as a solo dev. The landing page says: Zero Telemetry— no outbound traffic, no data collection Words are cheap, and that's just a claim . To a user, this is "an unknown solo dev's app that rewrites your firewall with root, saying 'trust me, I send nothing.'" There's no reason to believe it. And this app sees sensitive stuff, constantly: the gateway MAC and fingerprint, listening ports, the Wi-Fi SSID you're on, FileVault / SIP / firewall state, files it scanned, URLs it checked. If any of that leaked, it would betray exactly the people who picked it for privacy. So I decided to turn the claim into a measurement. tcpdump and free tools only — wiretap my own app. This post is the method and the places I tripped. "The MCP server and detection logic are open source — just read them." A few holes: 1. The main app isn't open source. Only the read-only MCP server and detection logic are public; the privileged helper, pf control, and licensing stay closed that surface helps attackers . Code alone doesn't cover the whole thing. 2. No App Sandbox. To be upfront: RoamSwitch does not run inside App Sandbox — it controls the firewall and sharing daemons, so it can't. The consequence is that entitlements don't restrict outbound traffic . codesign -d --entitlements showing "no network-client key" is not proof it can't send. You have to show behaviour. 3. Regressions. Even if this build is truly silent, a future version could add an analytics SDK, a crash reporter, a "check for news" feature. A one-time check is worthless — this needs to be a procedure you can re-run every release . So instead of "read the code," I hand people "run this procedure." The only traffic allowed out is the four paths documented in the whitepaper §7 : | | destination | process | when | |---|---|---|---| | 1 | lafine.net/api/v1/license/ | RoamSwitch | license activate / deactivate only | | 2 | lafine.net/updates/appcast.xml | RoamSwitch | Sparkle launch + every 24 h | | 3 | ClamAV mirrors | freshclam not the app | only with ClamAV installed | | 4 | HEAD to a target URL | RoamSwitch | only when the in-app Link Safety sheet is used | Anything else out of RoamSwitch / RoamSwitchHelper / RoamSwitchMCPServer is a fail . I don't own Little Snitch, so: brew install wireshark — pull destinations and TLS SNI from the pcapLuLu is a fine Little Snitch substitute, and it's by a well-known name in Mac security — which, for a write-up, only helps. I'd convinced myself that "Sparkle's periodic check is every 24 h, so I have to capture for 25 h straight to span it." I set up for an overnight run. Then I looked at Info.plist : SUScheduledCheckInterval = 86400 , and SULastCheckTime sits in defaults in the clear. So there's no waiting: BID=com.tetsuharu.RoamSwitch defaults delete $BID SULastCheckTime osascript -e 'quit app "RoamSwitch"'; sleep 2; open -a RoamSwitch launch sees "overdue" and checks within seconds to a couple of minutes The appcast HTTPS GET is byte-for-byte identical whether it comes from this, the "Check for Updates" menu item, or a fresh first launch. The only reason to run long is to catch a wall-clock "once a day" telemetry beacon — an unusual design, and there's no analytics SDK in the app anyway open source, checkable . So a few hours of idle is plenty. Total active work: ~3–4 h, one sitting. tcpdump tells you "a SYN went to 140.82.112.x." It does not tell you whether RoamSwitch, Chrome, Dropbox, or the OS sent it. On a daily-driver machine, even idle, there's a flood of non-Apple traffic: Safe Browsing, component updaters, cloud sync, chat keep-alives. Picking RoamSwitch's packets out of that by timestamp and port is genuinely error-prone — you end up waving things away as "probably Chrome." So you make the machine silent first. A dedicated account or spare Mac is ideal; quieting the same login also works: launchctl bootout the third-party agents reverts on reboot softwareupdate --schedule off , analytics off, Spotlight suggestions offThen, before launching RoamSwitch, run tcpdump for 10 minutes and check that every remaining destination is Apple 17.0.0.0/8 , etc. . If a non-Apple destination is still there, something's still running — find it with lsof -nP -iTCP@<IP , stop it, start over. Only once that baseline is clean can you lean on "any non-Apple traffic after this ≈ RoamSwitch." Don't open this procedure in a browser on the test machine. Chrome especially is a pile of non-Apple traffic Safe Browsing, a GCM keep-alive . Use a second device, print it, or pre-cache the page and use Safari. Reproduce the situation RoamSwitch is built for: you join a café / public Wi-Fi and the app locks down automatically. At max lockdown, RoamSwitch's pf blocks other apps' egress too, so the capture is even quieter and its own traffic stands out. Crucially, appcast and license still get through — RoamSwitch puts its own traffic in a pf exception. Part of the point of the test is to confirm that exception is minimal and only points where it's supposed to i.e. it didn't quietly whitelist a telemetry endpoint for itself . layer 1: full capture local / DNS / mDNS filtered out sudo tcpdump -i any -n -w cap-%Y%m%d-%H%M.pcap -G 3600 -W 72 -Z root \ ' ip or ip6 and not net 127.0.0.0/8 and not net 169.254.0.0/16 \ and not net 224.0.0.0/4 and not port 5353 and not port 53 and not port 67 and not port 68' layer 2: socket - process, sampled every 5s, in a root shell sudo bash -c 'while :; do ts=$ date -u +%FT%TZ ; \ lsof -nP -iTCP -sTCP:ESTABLISHED +c0 2 /dev/null \ | awk -v ts="$ts" "NR 1{print ts,\$1,\$2,\$9}"; sleep 5; done' | tee lsof.log layer 3: LuLu events from the unified log log stream --style syslog --predicate 'subsystem BEGINSWITH "com.objective-see"' lulu.log & The MCP server only speaks over stdio, so I drive every tool through it and separately confirm its PID never opens an outbound socket. After the idle hold and the phases run every feature by hand once; deliberately fire the four known paths , pull destinations from the pcap and cross-reference with the process log by timestamp: for f in cap- .pcap; do tshark -r "$f" -Y 'tcp.flags.syn==1 && tcp.flags.ack==0' -T fields -e ip.dst done | sort -u awk '$2 ~ /^RoamSwitch/ {for i=1;i<=NF;i++ if $i ~ /- / {split $i,a,"- " ; print $2,a 2 }}' \ lsof.log | grep -vE '127\.0\.0\.1|::1' | sort -u Pass criteria: RoamSwitchHelper — RoamSwitchMCPServer — RoamSwitch — nothing outside lafine.net and the URL you hit for path 4 freshclam , not the appVerdict: PASS . Across a ~2-hour capture forced Sparkle check, every feature exercised once, DNS Threat Guard toggled, a network transition, security level pinned to Maximum Lockdown : Zero non-local outbound connections attributed to RoamSwitch, RoamSwitchHelper, or RoamSwitchMCPServer. The only sockets the MCP server opened were 127.0.0.1:5000 and 127.0.0.1:7000 — the get exposed ports tool checking HTTP headers on locally-listening dev servers. Nothing left the machine. Here is every HTTPS destination in the whole capture TLS SNI : bag.itunes.apple.com gspe35-ssl.ls.apple.com proxy-safebrowsing.googleapis.com configuration.apple.com init.push.apple.com setup.icloud.com dns.quad9.net mask.icloud.com swdist.apple.com gateway.icloud.com mesu.apple.com swscan.apple.com gsa.apple.com ocsp2.apple.com tether.edge.apple p117-contacts.icloud.com courier.push.apple.com swallow.apple.com fbs.smoot.apple.com lafine.net <- the only non-OS destination Strip out the OS's own services iCloud, push, software update, OCSP, Safari Safe Browsing and one destination remains: lafine.net — Sparkle fetching appcast.xml egress path 2 . dns.quad9.net shows up because the DNS Threat Guard switched the system resolver; the encrypted lookups are mDNSResponder 's, not RoamSwitch's. swallow.apple.com / fbs.smoot.apple.com sit on AWS IPs but are Apple services. entitlements: bash $ codesign -d --entitlements :- /Applications/RoamSwitch.app/Contents/MacOS/RoamSwitchHelper <dict </dict $ codesign -d --entitlements :- /Applications/RoamSwitch.app/Contents/MacOS/RoamSwitchMCPServer <dict </dict Empty — not even com.apple.security.network.client which, as noted, wouldn't enforce anything without App Sandbox, hence the behavioural check . LuLu logged no RoamSwitch-related outbound events for the duration. It's all in one shell script, in the audit/ folder of the roamswitch-support repo. ./rs-zerotel-audit.sh all --idle 2h --clam runs prep, capture, forced Sparkle check, MCP exercise, analysis, a PASS/FAIL verdict, and a shareable FINDINGS.md . The only manual bits are helper approval, pinning max lockdown, one network transition a Wi-Fi off/on is enough , and a few menu actions. Being upfront: But "a claim" versus "a claim + a procedure anyone can run + an honest statement of the limits" is a very different weight of trust. The more skeptical someone is, the more they want something they can check themselves. Script and run sheets EN/JA : github.com/lafine1211/roamswitch-support, audit/