I Trained a Model to Predict Machine Failure, Then Spent Most of My Time on Everything After the… A developer spent one day training a machine learning model to predict machine failure but considerably longer turning it into production-ready software, reporting that the model achieved 74.20% accuracy with feature selection on a synthetic dataset of 5,000 samples. The final system, deployed via Docker and Kubernetes, exposes a POST /predict endpoint that returns failure probability and risk category, with a high-risk example yielding 0.9439 probability. The developer emphasized that the real challenge was not the ML but the surrounding engineering: automated testing, versioning, health checks, and deployment. The machine learning part of this project took a day. Getting that model into something that actually behaves like production software, tested automatically, versioned, deployed with health checks Kubernetes actually respects, took considerably longer. That gap is the real subject here. Predictive maintenance means using a machine’s operational data, temperature, vibration, pressure, and so on, to catch failure conditions before they become failures. The dataset here was synthetic industrial telemetry: 5,000 samples, eight input features, one binary target. No missing values, and a reasonably balanced split, 54.24% no-failure versus 45.76% failure, which avoided the severe class imbalance I’d run into on an earlier server-health experiment. Before touching a model, I checked which features correlated with the failure label at all. Vibration came out on top at 0.31, followed by operating hours at 0.25 and temperature at 0.25. Rotational speed and pressure barely mattered, 0.003 and -0.02 respectively. That ordering held up consistently once actual models were trained, which is a small but useful confirmation that the correlation check wasn’t just noise. Logistic Regression, run through a StandardScaler pipeline with an 80/20 stratified split, landed at 73.90% accuracy, 72.64% precision, 69.00% recall, and an F1 score of 70.77%. Its standardized coefficients agreed with the correlation analysis: vibration 0.88 , temperature 0.69 , and operating hours 0.67 carried the most weight, while maintenance count pulled in the opposite direction at -0.58, more past maintenance events correlated with lower predicted failure risk, which is exactly the direction you'd hope for. Random Forest, with 200 estimators, came in slightly behind at 71.20% accuracy and an F1 score of 67.93%. Its feature importances told the same story from a different angle: vibration, operating hours, and temperature again at the top, though closer together than the logistic regression coefficients suggested, at 0.186, 0.160, and 0.153. A feature-selection pass using just the strongest predictors edged out both, 74.20% accuracy, 71.08% F1. Small gain, but a real one, and it’s the version that ended up serialized for inference. None of these numbers are dramatic. They’re honest, and for a synthetic dataset built to have real signal in it, that felt like the right outcome to report rather than inflate. POST /predict takes machine telemetry and returns a failure prediction, a probability, and a risk category, LOW, MEDIUM, or HIGH, rather than a bare 0 or 1. A high-risk example returned "failure probability": 0.9439, "risk": "HIGH". A low-risk one returned 0.014. That probability is what actually makes the endpoint useful; a binary flag alone throws away exactly the information a maintenance team would want when deciding how urgently to act. The image runs on a python:3.12-slim base, installs dependencies, copies the app and the serialized model, and exposes port 8000 running Uvicorn. Before any of it reached Kubernetes, it got tested locally: start the container, hit /health, confirm {"status": "healthy"}, then test /predict with real payloads. Skipping that step and debugging Kubernetes-level networking issues on top of application-level ones at the same time is a genuinely bad way to spend an afternoon. GitHub Actions builds the image, starts it as a container, waits for startup, and calls /health with retry logic before deciding the build is valid. That retry logic exists because the first version of this pipeline didn't have it, and failed with curl: 56 Recv failure: Connection reset by peer, not because anything was actually broken, but because the application needed more time to finish starting than the health check gave it credit for. docker build succeeding was never the bar. The container actually answering requests was. Once validation passes, the image gets pushed to Docker Hub under two tags: latest, and the specific Git commit SHA, e.g. ikramsyed/predictive-maintenance:44d3b6116d844a150a9cf67d6998ed4a6acd88a4. The SHA tag means any running container can be traced back to the exact source revision that produced it, which latest alone can never guarantee. On first deploy, the readiness probe failed with connect: connection refused, because the FastAPI app hadn't started listening on port 8000 yet. The important part is what didn't happen: the container wasn't restarted. Kubernetes just kept the pod out of Service rotation until it actually reported ready, Ready: True, Restart Count: 0, once startup finished. That's the entire distinction between a readiness probe and a liveness probe made concrete: temporarily not ready is not the same failure as actually broken, and treating them the same would mean restarting a pod for simply taking a few seconds to start. Each pod requests 200m CPU and 256Mi memory, with limits of 400m and 512Mi. Across two replicas, that’s a combined request of 400m CPU and 512Mi memory, with a theoretical ceiling of 800m and 1Gi. The resulting QoS class was Burstable, not Guaranteed, since requests and limits don’t match exactly. That’s a deliberate middle ground: enough headroom for the app to burst under load, without the ability to consume the entire node’s resources unchecked. The NGINX Ingress Controller showed EXTERNAL-IP: