{"slug": "engineering-journey-fine-tuning-llms-from-laptop-to-production", "title": "Engineering Journey: Fine-Tuning LLMs from Laptop to Production", "summary": "An engineer detailed a journey fine-tuning LLMs from a local Apple Silicon setup to production on AWS SageMaker, reporting that moving training from an M4 Mac Mini to a SageMaker ml.g4dn.xlarge spot instance cut a 14-hour MLX run to 3.3 hours with roughly 60% cost savings. The project used MLX, MLflow, DVC, and SageMaker Pipelines V2 to achieve reproducible, auditable training, with a ConditionStep promoting models only when test loss improved.", "body_md": "The project started simply. I needed a fine-tuned math reasoning model and wanted to prove it was possible.\n\nI reached for **MLX** — Apple’s ML framework for Silicon — because my M4 Mac Mini was the fastest thing I had. mlx_lm Made LoRA fine-tuning straightforward: load the model, freeze everything except the adapter layers, run a training loop. I had a Qwen2.5-1.5B adapter with val loss dropping from 1.8 to 0.9 on MathInstruct.\n\nThe next problem was immediate: I ran six experiments and lost track of which adapter came from which hyperparameters. So I added **MLflow** for experiment tracking and **DVC** for data versioning.\n\nDVC turned out to be the most important early decision. The core idea is simple — instead of tracking dataset files in git, DVC stores a content hash in dvc.lock and keeps the actual bytes in a remote cache (S3). That hash, combined with the git commit SHA, becomes a permanent fingerprint for every training run:\n\n```\ngit_sha + dvc.lock_MD5  →  \"e90324a9-dvc3f2a1b7c\"\n```\n\nThis string became the dvc.commit tag on every MLflow run. From it, I can always reconstruct the exact state: git checkout <sha> && dvc checkout. That guarantee — not just \"I think I used the same data\" but provably the same data — carried forward into every pipeline iteration.\n\n**What worked:** Rapid iteration, zero infrastructure cost, and a lineage model that scaled to the cloud pipelines that followed.\n\n**What broke:** Training on the full MathInstruct dataset (260k examples) took 14 hours on the M2. That was the ceiling. And there was no path to multi-GPU training or team collaboration from a local MLX setup.\n\n**Repo**: [lora-finetune-mlx](https://github.com/ArkaSanka/lora-finetune-mlx)\n\nThe move to the cloud was forced by three converging pressures: training time, environment reproducibility, and the need for a proper pipeline instead of a collection of scripts.\n\nI chose **SageMaker Pipelines V2** because it gave me exactly what was missing: a managed DAG with caching, conditional branching, and a visual execution graph. I designed a five-step pipeline:\n\n```\nPrepareData → TrainLoRA → Evaluate → ReadEvalReport → CheckTestLoss → RegisterModel\n```\n\nThe ConditionStep at the end is the key production-grade feature — the model only gets promoted to the MLflow registry if test loss actually improved. No manual approval gate, no accidentally shipping a regression.\n\nTraining on ml.g4dn.xlarge spot reduced a 14-hour MLX run to **3.3 hours**, with spot savings of roughly 60% over on-demand. The checkpoint-to-S3 mechanism meant a spot interruption cost five minutes of lost progress, not the entire run.\n\nThe bugs I hit were real and expensive to diagnose:\n\n**processing_class vs** **tokenizer in SFTTrainer.** The HuggingFace DLC for PyTorch 2.3 ships with trl < 0.12. My script crashed 30 minutes into the first run at a TypeError I didn't expect. Fix: use tokenizer= for backward compatibility.\n\n**KeyError: 'ModelArtifacts' after Ctrl+C.** When you interrupt a SageMaker job, estimator.model_data raises KeyError because the model hasn't been packaged yet. I wrote an explicit _wait_training() polling loop instead of relying on the SDK.\n\n**PropertyFile only works with** **ProcessingStep.** My evaluation ran as a TrainingStep (to use the GPU), but ConditionStep requires a PropertyFile which only attaches to ProcessingStep outputs. I had to add a tiny relay step (ReadEvalReport) just to bridge the metric. The SageMaker docs don't cover this combination clearly.\n\nEach of these bugs cost 30–60 minutes of compute time to reproduce and diagnose. I documented every fix so the next engineer doesn’t pay the same tax.\n\nThe DVC lineage pattern from Chapter 1 carried forward unchanged. The git+DVC hash became a DVCCommitHash SageMaker Pipeline parameter — indexed in the SageMaker Lineage Graph — and logged as a dvc.commit MLflow tag inside the container. Two sources of truth for the same fact.\n\n**What it bought:** Reproducible, auditable, cost-optimized single-node training with conditional model promotion. The right pipeline for production when one GPU is enough.\n\n**What it couldn’t do:** Scale to multiple instances. SFTTrainer is a single-process trainer. Getting to multi-GPU required a different training stack.\n\n**Repo**: [lora-finetune-sagemaker](https://github.com/ArkaSanka/lora-finetune-sagemaker)\n\nTwo things pushed me to build the Ray pipeline as a parallel track:\n\nFirst, we started exploring 7B models. A single ml.g4dn.xlarge (16GB VRAM) isn't enough for a 7B model at full precision. I needed data parallelism across instances without rewriting the entire training loop.\n\nSecond, I wanted better observability during training. The V2 pipeline gives you CloudWatch system metrics after the fact. Ray adds a live Dashboard showing per-worker task graphs, OOM events, and gradient sync timing — and TensorBoard events stream to S3 during training via TensorBoardOutputConfig. Watching loss curves update in real-time during a 3-hour job changes how you debug.\n\nThe architectural difference from the V2 pipeline is intentional:\n\n```\nSM V2 Pipeline:  SageMaker orchestrates the DAG → SFTTrainer runs (single process)Ray pipeline:    Python chains the jobs → SageMaker provisions → Ray coordinates workers\n```\n\nSageMaker handles instance provisioning and spot recovery. Ray handles worker coordination inside those instances. The training scripts have zero SageMaker SDK imports — pure Ray + HuggingFace — which makes them testable locally and portable to other compute backends.\n\nThe bug that cost the most time: **Ray Train worker** **cwd is not** **/opt/ml/code/**. Ray spawns workers in subprocesses with their own working directory. A relative params.yaml path worked on the head node and failed silently on every worker. I traced it through CloudWatch logs to a FileNotFoundError on worker 1, 45 minutes into the first run. Fix: Path(__file__).parent / args.config Resolved to absolute before passing to train_loop_config.\n\nScaling data parallelism is a single flag:\n\n```\npython -m ray_pipeline.launch --num-instances 2   # 2 workers, LR auto-scaled 2×python -m ray_pipeline.launch --num-instances 4   # 4 workers, LR auto-scaled 4×\n```\n\nThe learning rate linear scaling rule (Goyal et al. 2017) is wired into the training script — doubling workers doubles the effective batch size, so LR doubles to maintain training dynamics.\n\nThe DVC lineage pattern is identical to the V2 pipeline. _dvc_commit_hash() in launch.py computes the same git+DVC fingerprint, passes it as a SageMaker hyperparameter, and the container logs it as a dvc.commit MLflow tag. Any model from any of the three pipelines can be traced back to its exact code and data version the same way.\n\n**What it bought:** True multi-node data parallelism, live TensorBoard monitoring, and a simpler orchestration model that’s easier to extend.\n\n**What it trades off:** No conditional DAG, no built-in model registry gating, no visual pipeline graph in SageMaker Studio.\n\n**Repo**: [lora-finetune-ray](https://github.com/ArkaSanka/lora-finetune-ray)\n\nThe same lineage fingerprint appears in every repo:\n\n```\ngit commit SHA ─┐                ├─→  \"e90324a9-dvc3f2a1b7c\"dvc.lock MD5  ─┘         │                          ├─→  MLflow tag: dvc.commit                          ├─→  SageMaker parameter: DVCCommitHash                          └─→  mlflow.log_input() dataset record\nReproduce any run from any pipeline:  git checkout e90324a9 && dvc checkout\n```\n\nThis isn’t accidental. I built it in from Chapter 1 because the cost of adding it later — retrofitting lineage to existing runs — is much higher than starting with it. Every interview question about reproducibility or compliance now has a concrete, working answer.\n\n[Engineering Journey: Fine-Tuning LLMs from Laptop to Production](https://pub.towardsai.net/engineering-journey-fine-tuning-llms-from-laptop-to-production-b1852f8ada39) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/engineering-journey-fine-tuning-llms-from-laptop-to-production", "canonical_source": "https://pub.towardsai.net/engineering-journey-fine-tuning-llms-from-laptop-to-production-b1852f8ada39?source=rss----98111c9905da---4", "published_at": "2026-09-08 21:01:01+00:00", "updated_at": "2026-09-08 21:29:25.761357+00:00", "lang": "en", "topics": ["machine-learning", "large-language-models", "mlops", "ai-infrastructure"], "entities": ["Apple", "MLX", "Qwen2.5-1.5B", "MathInstruct", "MLflow", "DVC", "AWS SageMaker", "Hugging Face"], "alternates": {"html": "https://wpnews.pro/news/engineering-journey-fine-tuning-llms-from-laptop-to-production", "markdown": "https://wpnews.pro/news/engineering-journey-fine-tuning-llms-from-laptop-to-production.md", "text": "https://wpnews.pro/news/engineering-journey-fine-tuning-llms-from-laptop-to-production.txt", "jsonld": "https://wpnews.pro/news/engineering-journey-fine-tuning-llms-from-laptop-to-production.jsonld"}}