{"slug": "tips-for-running-stable-background-ml-inference-on-macos", "title": "Tips for Running Stable Background ML Inference on macOS", "summary": "A developer from Workstyle Tech shared tips for running background ML inference services on macOS, highlighting pitfalls such as missing GNU coreutils commands and locale-related errors. The post recommends using nohup and disown for backgrounding, setting LC_ALL=C to avoid 'Illegal byte sequence' errors in log processing, and polling health endpoints instead of using fixed sleeps. These practices aim to improve reliability of startup scripts for services like FastAPI and uvicorn.", "body_md": "📝 Originally published (in Japanese) at\n\n[forge.workstyle.tech].\n\nRunning an inference service as a background process on macOS, with a Linux server mindset, can lead to subtle issues. Things like \"a one-liner that works on Linux doesn't work on Mac\" or \"grepping logs results in garbled text errors and crashes\" — these are minor but time-consuming problems.\n\nThis article compiles a collection of short tips gathered from running a Seed-VC based voice conversion service (FastAPI + uvicorn, local `127.0.0.1:8770`\n\n) as a background process on macOS. It focuses on macOS-specific pitfalls not covered in Linux-centric articles.\n\n`setsid`\n\n/ `timeout`\n\nare not available on macOS\nFirst, it's important to note that **macOS (BSD-based) does not include GNU coreutils' setsid or timeout by default**. If you use these commands, which are used for backgrounding and timed execution on Linux, directly in a Mac script, you'll get a\n\n`command not found`\n\nerror.There are two solutions:\n\n`coreutils`\n\nvia Homebrew and use `gsetsid`\n\n/`gtimeout`\n\nFor background scripts that avoid external dependencies, using standard tools as alternatives is a safer option.\n\n`nohup`\n\n+ `disown`\n\nfor background processes\nIn environments without `setsid`\n\n, the combination of `nohup`\n\nand `disown`\n\nis reliable for keeping processes alive even after closing the shell.\n\n```\n# Run the inference service as a background process\nnohup bash scripts/start-backend.sh > backend.log 2>&1 &\ndisown\n```\n\n`nohup`\n\n... Ignores hangup signals (SIGHUP), keeping the process alive even after the terminal is closed`> backend.log 2>&1`\n\n... Redirects standard output and standard error to a log file`&`\n\n... Runs the process in the background`disown`\n\n... Removes the job from the shell's job table, preventing it from being terminated when the shell is closedWhile `nohup`\n\nalone usually keeps the process alive, adding `disown`\n\nensures that closing the terminal won't accidentally terminate the process.\n\n`tr`\n\n/ `grep`\n\nfail with binary data in logs → use `LC_ALL=C`\n\nThis was the most problematic issue on Mac. Inference logs may contain progress bar control characters or, occasionally, garbled multibyte sequences. When processed by macOS's `tr`\n\nor `grep`\n\n, you'll see:\n\n`tr: Illegal byte sequence`\n\nThis happens because the locale is set to UTF-8, causing invalid byte sequences to be treated as \"invalid characters\" and throwing an exception.\n\nThe solution is to set the locale to **C (pass-through as byte sequences)** for those commands.\n\n```\n# Remove unwanted control characters from logs (avoids Illegal byte sequence)\nLC_ALL=C tr -d '\\r' < backend.log > backend.clean.log\nLC_ALL=C grep \"ERROR\" backend.log\n```\n\nSetting `LC_ALL=C`\n\ntreats text as \"bytes\" rather than \"characters,\" preventing crashes due to invalid sequences. This is safer for pipelines that process or search logs programmatically.\n\nLoading models takes time, so sending requests immediately after starting with `nohup`\n\nwill fail because the service isn't ready. Using `sleep 10`\n\nas a workaround is unreliable—too short for slow machines and too long for fast ones.\n\nThe proper approach is to **poll the service's health endpoint until it returns a 200 status**. In this setup, the health endpoint is `http://127.0.0.1:8770/health`\n\n, so we poll it.\n\n```\n# Wait for the health endpoint to be ready before proceeding\nuntil curl -sf http://127.0.0.1:8770/health >/dev/null; do\n  sleep 1\ndone\necho \"backend ready\"\n```\n\n`curl -sf`\n\nexits with a non-zero status on failure, so combining it with `until`\n\nallows you to wait until the service is ready. Waiting based on **state**, not a fixed delay, significantly improves the reliability of startup scripts.\n\n`pkill`\n\nusing pattern matching\nFor background processes without a saved PID, `pkill`\n\nwith a command-line pattern is convenient for stopping them.\n\n```\n# Stop the inference service started with uvicorn\npkill -f \"uvicorn server:app\"\n```\n\nThe `-f`\n\noption matches the entire command line, so including specific details like the port or app name in the pattern prevents unrelated processes from being affected. For more precision, save the PID during startup and target it directly.\n\n`setsid`\n\n/`timeout`\n\n`coreutils`\n\n(`gsetsid`\n\n/`gtimeout`\n\n) or use standard tool alternatives`nohup ... & disown`\n\n`tr`\n\n/`grep`\n\nto fail with `Illegal byte sequence`\n\n`LC_ALL=C`\n\n`sleep`\n\ndelays, `until curl -sf`\n\n`pkill -f \"specific pattern\"`", "url": "https://wpnews.pro/news/tips-for-running-stable-background-ml-inference-on-macos", "canonical_source": "https://dev.to/orca_forge/tips-for-running-stable-background-ml-inference-on-macos-26dc", "published_at": "2026-08-04 01:08:21+00:00", "updated_at": "2026-08-04 01:38:59.145848+00:00", "lang": "en", "topics": ["machine-learning", "developer-tools", "mlops"], "entities": ["Workstyle Tech", "Seed-VC", "FastAPI", "uvicorn", "macOS", "Homebrew"], "alternates": {"html": "https://wpnews.pro/news/tips-for-running-stable-background-ml-inference-on-macos", "markdown": "https://wpnews.pro/news/tips-for-running-stable-background-ml-inference-on-macos.md", "text": "https://wpnews.pro/news/tips-for-running-stable-background-ml-inference-on-macos.txt", "jsonld": "https://wpnews.pro/news/tips-for-running-stable-background-ml-inference-on-macos.jsonld"}}