{"slug": "wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx", "title": "WiFi off, model running: what broke (and what shipped) building YOLO26 on MLX", "summary": "A two-person team built SENTINEL, a single-file, fully offline on-device posture attention map that runs YOLO26n at roughly 16 FPS at 720p on an M4 Mac with WiFi physically disabled. Engineer Igor Eduardo and field specialist Lexi Armstrong documented three failure modes encountered during the seven-day webAI YOLO26 MLX Build Challenge: a SIGTRAP crash in OpenCV 4.13's AVFoundation capture when frames are grabbed off the main thread on macOS 26, a silent hang caused by lazy MLX proxies in yolo-mlx's Boxes that deadlock when materialized in a multi-threaded pipeline, and a fine-tuning run on labeled posture data that failed to converge and was dropped. The shipped version uses released yolo26n weights plus a simple bounding-box geometry heuristic for standing, sitting, and lying classification.", "body_md": "**By [Igor Eduardo](https://igoreduardo.com) · Austin, TX · with Lexi Armstrong**\n\nSite: [igoreduardo.com](https://igoreduardo.com) · Repo: [github.com/nomad-link-id/sentinel-mlx](https://github.com/nomad-link-id/sentinel-mlx) · Demo: [youtu.be/c2v5Mdg5fpw](https://youtu.be/c2v5Mdg5fpw)\n\nThis is a build note from the webAI YOLO26 MLX Build Challenge (May 2026), not a product pitch. We shipped a single-file, on-device posture attention map that runs with WiFi physically off. The useful part for other builders is what failed first.\n\nSeven days. Two-person build: Lexi Armstrong owned the problem and operational constraints; I owned the engineering.\n\nWhat shipped uses **yolo26n** with released weights as-is, plus a deliberately simple bounding-box geometry heuristic for posture — standing / sitting / lying. Single-file Python (~200 LOC), single-thread synchronous loop, ~16 FPS at 720p on M4 / 32 GB / macOS 26.3 / Python 3.14, ~45 ms inference per frame.\n\nAn earlier attempt to fine-tune yolo26n on labeled posture data did not converge and was dropped.\n\nWe're writing this up because the failure modes are probably more useful than the demo — specifically the macOS 26 AVFoundation crash, the lazy-eval gotcha in yolo-mlx's `Boxes`, and the training collapse. And because the part that turned a demo into something a responder might actually trust came from the field side, not the code.\n\nSENTINEL is the meeting of two views of the same problem. Lexi saw it from the field — denied-comms / industrial security / operational-edge constraints, including what \"zero network egress\" has to mean when operational security depends on it. I saw it from healthtech — production-class triage and patient-flow systems in extended pilot with tier-1 hospitals in São Paulo. The core problem of who needs attention first when there are too many patients and not enough hands shows up in mass-casualty triage too, at different scale and stakes.\n\nConcretely, the field side drove:\n\nHonest division of labor: Lexi made sure it was worth building and aimed at the right target; I built it.\n\nThe initial architecture was a FastAPI backend exposing a WebSocket stream to a browser overlay, with camera capture in a `ThreadPoolExecutor`. Two hard blockers killed it.\n\nWith OpenCV 4.13's AVFoundation backend, capturing frames from a non-main thread crashed reliably with `SIGTRAP`. The trace pointed at:\n\n``` php\ncv2.abi3.so -> CaptureDelegate captureOutput -> CFRelease\n```\n\nThis looks like a CFRelease reference-counting issue when `AVCaptureSession` is owned outside the main thread on macOS 26. We didn't patch around it — rewriting the capture stack wasn't worth the risk in a 7-day window. Flagging it because anyone building on this stack with a worker-thread capture pattern is going to hit it.\n\n`Boxes` proxies\nThe harder one to isolate. Benchmark showed clean FPS. Warmup completed. Model loaded. But when the WebSocket handler called `detector.predict()` on a real camera frame, it hung silently. Every time.\n\nDiagnostic path:\n\n`dets = []`) — video appeared immediately. Camera, WebSocket, frontend all confirmed working.`predictor.py`: `_predict` start, `mx.eval` done, returning results all appeared.`predict()` in `detector.py`: `BEFORE PREDICT` appeared, `AFTER PREDICT` never did.\nThe hang was **not** in MLX inference itself — it was in the box-iteration loop after the model returned:\n\n```\nfor box in results[0].boxes:\n    x1, y1, x2, y2 = [int(v) for v in box.xyxy[0].tolist()]  # hangs here\n```\n\nRoot cause hypothesis: yolo-mlx's `Boxes` returns lazy MLX proxies for `.xyxy`, `.conf`, `.cls`. They aren't evaluated during inference — they're deferred. Calling `.tolist()` or indexing them triggers a secondary `mx.eval()` that deadlocked in the multi-thread setup. Warmup used `np.zeros` (zero detections), so the box loop never ran during warmup. Real frames produced detections, the loop ran for the first time, the lazy eval fired, and the pipeline froze.\n\nWorkaround — force-materialize before iterating:\n\n``` python\nimport mlx.core as mx\nimport numpy as np\n\nboxes = results[0].boxes\nmx.eval(boxes.xyxy); mx.eval(boxes.conf); mx.eval(boxes.cls)\nxyxy = np.array(boxes.xyxy)\nconf = np.array(boxes.conf)\ncls = np.array(boxes.cls)\n\nfor i in range(len(xyxy)):\n    x1, y1, x2, y2 = [int(v) for v in xyxy[i]]\n```\n\nNo `.tolist()` on MLX proxies inside loops. Convert to NumPy once, then iterate. Warmup with synthetic zero-detection frames does not exercise the box-iteration path, so this bug is invisible until a real subject enters the frame — worth a doc note or an explicit `.materialize()` helper on `Boxes`.\n\nAround day four we made a call. The architecture being hardened solved for production complexity not needed for a 7-day demo. We forked the official Yolo26-mlx challenge starter, deleted everything around the inference call, and rewrote it as a single synchronous loop on the main thread:\n\n```\nwhile True:\n    ok, frame = cap.read()\n    detections = model(frame)\n    overlay = render_sentinel_ui(frame, detections)\n    cv2.imshow(\"SENTINEL\", overlay)\n    if cv2.waitKey(1) & 0xFF == ord(\"q\"):\n        break\n```\n\nThat sidesteps the AVFoundation issue (capture on main thread) and the lazy-eval issue (no async / threading). ~16 FPS at 720p on M4 once settled. That's what shipped to the public repo.\n\nIn parallel, we tried fine-tuning yolo26n on labeled posture data — a Roboflow posture-classification dataset (`person_lying` / `person_sitting` / `person_standing`). Training loss collapsed to near-zero by epoch 2; mAP stuck at 0.0 for the remaining epochs. We didn't isolate the root cause in the time available — most likely a label-format mismatch or normalization gap — but those debugging cycles weren't available with AVFoundation and lazy-eval also live.\n\nNoting this because \"trained posture head\" appears on the roadmap, and we want to be explicit that it's an honest open problem — not something we skipped by choice.\n\nThe shipped version **does not** use a trained posture classifier, and SENTINEL **does not** make a clinical triage decision. It uses released `yolo26n.npz` weights as-is. The layer on top is a deterministic posture heuristic on bounding-box geometry. It reports what the camera can actually see — body posture — not medical severity.\n\n`class == 0` (person), `conf ≥ 0.40`\n`aspect = bbox_height / bbox_width`\n`aspect > 1.6` → standing`1.0 ≤ aspect ≤ 1.6` → sitting / slumped`aspect < 1.0` → lying down\nWhy posture, not severity: a camera cannot see a pulse, internal bleeding, or a blocked airway. The honest output is \"who is upright, who is down, and for how long\" — a visual cue that helps a responder decide where to look first. The human triages.\n\nWhere the heuristic fails (honest):\n\n\"We can write the rule on a napkin\" was the right call for a 7-day safety-relevant demo — but it's a starting point, not the answer.\n\n**Worked:** yolo-mlx 0.3.1 API; MLX Metal backend on Apple Silicon; starter repo structure (pivot took an afternoon); macOS 26 + Python 3.14 + M4 once threading was off the table.\n\n**Friction:**\n\n`??`.\nForm factor: laptop demo ≠ product. Realistic V1 path we sketched — Vision Pro for prototype, field-grade waveguide for first-responder pilot — both keep inference on-device. Platforms that move compute to a paired puck reintroduce a network leg that weakens the zero-egress story for this use case.\n\nWe didn't build SENTINEL to chase a market — we built it because the environments where attention allocation matters most are often the ones where the network isn't there. Before this is a product: trained posture classifier, a real labeled dataset, and validation with a real responder who either uses the map or ignores it. No projections — we haven't earned them yet.\n\nIf the AVFoundation crash, the lazy-eval workaround, or the training-collapse note is useful to YOLO26-MLX docs, happy to write more on any section.\n\n— **Igor Eduardo** ([igoreduardo.com](https://igoreduardo.com)) & Lexi Armstrong\n\nRepo: [nomad-link-id/sentinel-mlx](https://github.com/nomad-link-id/sentinel-mlx) · Demo: [youtu.be/c2v5Mdg5fpw](https://youtu.be/c2v5Mdg5fpw)\n\n*Disclosure: drafted with AI assistance from build notes; technical claims and wording owned by the authors.*", "url": "https://wpnews.pro/news/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx", "canonical_source": "https://dev.to/nomad-link-id/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx-j1f", "published_at": "2026-09-17 14:45:09+00:00", "updated_at": "2026-09-17 14:52:50.933448+00:00", "lang": "en", "topics": ["computer-vision", "ai-tools", "developer-tools", "machine-learning"], "entities": ["Igor Eduardo", "Lexi Armstrong", "SENTINEL", "YOLO26", "MLX", "yolo-mlx", "OpenCV", "FastAPI"], "alternates": {"html": "https://wpnews.pro/news/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx", "markdown": "https://wpnews.pro/news/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx.md", "text": "https://wpnews.pro/news/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx.txt", "jsonld": "https://wpnews.pro/news/wifi-off-model-running-what-broke-and-what-shipped-building-yolo26-on-mlx.jsonld"}}