Tips for Running Stable Background ML Inference on macOS

A developer from Workstyle Tech shared tips for running background ML inference services on macOS, highlighting pitfalls such as missing GNU coreutils commands and locale-related errors. The post recommends using nohup and disown for backgrounding, setting LC_ALL=C to avoid 'Illegal byte sequence' errors in log processing, and polling health endpoints instead of using fixed sleeps. These practices aim to improve reliability of startup scripts for services like FastAPI and uvicorn.

📝 Originally published in Japanese at forge.workstyle.tech . Running an inference service as a background process on macOS, with a Linux server mindset, can lead to subtle issues. Things like "a one-liner that works on Linux doesn't work on Mac" or "grepping logs results in garbled text errors and crashes" — these are minor but time-consuming problems. This article compiles a collection of short tips gathered from running a Seed-VC based voice conversion service FastAPI + uvicorn, local 127.0.0.1:8770 as a background process on macOS. It focuses on macOS-specific pitfalls not covered in Linux-centric articles. setsid / timeout are not available on macOS First, it's important to note that macOS BSD-based does not include GNU coreutils' setsid or timeout by default . If you use these commands, which are used for backgrounding and timed execution on Linux, directly in a Mac script, you'll get a command not found error.There are two solutions: coreutils via Homebrew and use gsetsid / gtimeout For background scripts that avoid external dependencies, using standard tools as alternatives is a safer option. nohup + disown for background processes In environments without setsid , the combination of nohup and disown is reliable for keeping processes alive even after closing the shell. Run the inference service as a background process nohup bash scripts/start-backend.sh backend.log 2 &1 & disown nohup ... Ignores hangup signals SIGHUP , keeping the process alive even after the terminal is closed backend.log 2 &1 ... Redirects standard output and standard error to a log file & ... Runs the process in the background disown ... Removes the job from the shell's job table, preventing it from being terminated when the shell is closedWhile nohup alone usually keeps the process alive, adding disown ensures that closing the terminal won't accidentally terminate the process. tr / grep fail with binary data in logs → use LC ALL=C This was the most problematic issue on Mac. Inference logs may contain progress bar control characters or, occasionally, garbled multibyte sequences. When processed by macOS's tr or grep , you'll see: tr: Illegal byte sequence This happens because the locale is set to UTF-8, causing invalid byte sequences to be treated as "invalid characters" and throwing an exception. The solution is to set the locale to C pass-through as byte sequences for those commands. Remove unwanted control characters from logs avoids Illegal byte sequence LC ALL=C tr -d '\r' < backend.log backend.clean.log LC ALL=C grep "ERROR" backend.log Setting LC ALL=C treats text as "bytes" rather than "characters," preventing crashes due to invalid sequences. This is safer for pipelines that process or search logs programmatically. Loading models takes time, so sending requests immediately after starting with nohup will fail because the service isn't ready. Using sleep 10 as a workaround is unreliable—too short for slow machines and too long for fast ones. The proper approach is to poll the service's health endpoint until it returns a 200 status . In this setup, the health endpoint is http://127.0.0.1:8770/health , so we poll it. Wait for the health endpoint to be ready before proceeding until curl -sf http://127.0.0.1:8770/health /dev/null; do sleep 1 done echo "backend ready" curl -sf exits with a non-zero status on failure, so combining it with until allows you to wait until the service is ready. Waiting based on state , not a fixed delay, significantly improves the reliability of startup scripts. pkill using pattern matching For background processes without a saved PID, pkill with a command-line pattern is convenient for stopping them. Stop the inference service started with uvicorn pkill -f "uvicorn server:app" The -f option matches the entire command line, so including specific details like the port or app name in the pattern prevents unrelated processes from being affected. For more precision, save the PID during startup and target it directly. setsid / timeout coreutils gsetsid / gtimeout or use standard tool alternatives nohup ... & disown tr / grep to fail with Illegal byte sequence LC ALL=C sleep delays, until curl -sf pkill -f "specific pattern"