{"slug": "stop-saying-it-works-on-my-machine-docker-for-ai-engineers", "title": "Stop Saying \"It Works on My Machine\": Docker for AI Engineers", "summary": "Docker solves the 'it works on my machine' problem for AI engineers by packaging all dependencies—OS, Python, packages, code, and config—into a single portable image. A typical AI project has multiple dependency layers (Python packages, system libraries, CUDA toolkits) that Docker captures entirely, ensuring reproducibility across laptops and cloud GPU servers. Key practices include pinning package versions and copying requirements.txt before code to leverage Docker's layer caching.", "body_md": "You trained the model. The notebook runs. The demo works. You push it to a teammate, and forty minutes later you get the message every engineer dreads:\n\n\"Hey, I'm getting a CUDA error. And\n\n`torch`\n\nwon't import. And what version of Python is this?\"\n\nAnd you say the seven words that have haunted software since the dawn of time:\n\n\"But it works on my machine.\"\n\nHere's the uncomfortable truth: *\"it works on my machine\"* isn't a defense. It's a confession. It means your code depends on something living on your laptop that you never wrote down a Python version, a system library, a CUDA toolkit, a stray environment variable, a model file sitting in `~/Downloads`\n\n.\n\nDocker is how you stop making that confession. Let's fix this.\n\nA typical web app has a handful of dependencies. An AI project has *layers* of them, and each layer can betray you:\n\n`torch`\n\n, `transformers`\n\n, `numpy`\n\n, and the version conflicts between them.`libgl1`\n\nor `ffmpeg`\n\nthat pip won't install for you.`requirements.txt`\n\ncaptures *one* of those five layers. Docker captures all of them. That's the whole pitch.\n\nForget the whale logo and the buzzwords for a second.\n\nA **Docker image** is a frozen snapshot of a complete computer: the operating system, Python, your packages, your code, and your config, all baked into one file. A **container** is a running copy of that snapshot.\n\nThe mental model that makes it click: a virtual machine simulates an entire computer including its own operating system kernel, which is heavy and slow. A container shares your machine's kernel and only packages everything *above* it. So it boots in seconds, not minutes, and a single image runs identically on your laptop, your teammate's laptop, and a cloud GPU server.\n\nYou write the recipe once. Everyone gets the exact same kitchen.\n\nA `Dockerfile`\n\nis just that recipe, a plain text file of instructions. Here's a real one for a PyTorch project, with every line explained:\n\n```\n# Start from an official Python image. The \"-slim\" variant is smaller.\nFROM python:3.11-slim\n\n# Install system libraries that pip can't. Many vision/audio\n# libraries need these, and forgetting them is a classic\n# \"works on my machine\" trap.\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n    build-essential \\\n    libgl1 \\\n    && rm -rf /var/lib/apt/lists/*\n\n# Set the working directory inside the container.\nWORKDIR /app\n\n# Copy ONLY requirements first, then install.\n# This is a caching trick, see the note below.\nCOPY requirements.txt .\nRUN pip install --no-cache-dir -r requirements.txt\n\n# Now copy the rest of your code.\nCOPY . .\n\n# The command that runs when the container starts.\nCMD [\"python\", \"predict.py\"]\n```\n\nTwo beginner mistakes this avoids:\n\n**1. Pin your versions.** Your `requirements.txt`\n\nshould look like this, not just bare package names:\n\n```\ntorch==2.3.1\ntransformers==4.41.2\nfastapi==0.111.0\nuvicorn==0.30.1\nnumpy==1.26.4\n```\n\n`torch`\n\nwithout a version is a future outage waiting to happen. The whole point of Docker is reproducibility, don't undermine it by letting versions float.\n\n**2. Copy requirements.txt before your code.** Docker builds in layers and caches each one. If you copy everything at once, changing a single line of code forces it to reinstall\n\n`torch`\n\n(a multi-minute download) every single build. By copying requirements first, Docker reuses the cached install layer and only re-runs steps that actually changed. Your build goes from minutes to seconds.To build and run it:\n\n```\ndocker build -t my-model .\ndocker run my-model\n```\n\nThat `-t my-model`\n\njust names the image. The `.`\n\ntells Docker to look for the `Dockerfile`\n\nin the current folder. That's it, you now have a portable, reproducible model.\n\nBeginners often `COPY model.bin`\n\nstraight into the image. Don't. A 5GB image is painful to build, push, and pull, and you'll rebuild it every time the weights change.\n\nInstead, keep large files *outside* the image and mount them at runtime with a **volume**, a shared folder between your machine and the container:\n\n```\ndocker run -v $(pwd)/models:/app/models my-model\n```\n\nThis maps your local `models/`\n\nfolder to `/app/models`\n\ninside the container. The weights live on disk, the image stays lean, and you can swap models without rebuilding anything.\n\nMost of the time you don't just want to run a script, you want a model behind an endpoint your app can call. Here's a minimal FastAPI server, `app.py`\n\n:\n\n``` python\nfrom fastapi import FastAPI\nfrom pydantic import BaseModel\nimport torch\n\napp = FastAPI()\n\n# Load the model ONCE at startup, not on every request.\n# This is the single biggest performance mistake beginners make.\nmodel = torch.load(\"/app/models/model.pt\", map_location=\"cpu\")\nmodel.eval()\n\nclass Request(BaseModel):\n    text: str\n\n@app.post(\"/predict\")\ndef predict(req: Request):\n    with torch.no_grad():\n        result = model(req.text)\n    return {\"prediction\": result}\n\n@app.get(\"/health\")\ndef health():\n    return {\"status\": \"ok\"}\n```\n\nNotice the model loads *once* when the server boots, not inside `predict()`\n\n. Loading weights on every request will make your API crawl, a mistake that's easy to miss until production traffic hits.\n\nNow adjust the Dockerfile's last line to launch the server instead of a script:\n\n```\nCMD [\"uvicorn\", \"app:app\", \"--host\", \"0.0.0.0\", \"--port\", \"8000\"]\n```\n\nThat `--host 0.0.0.0`\n\nmatters. Inside a container, the default `127.0.0.1`\n\nmeans \"only reachable from inside this container\", your requests from outside would bounce. Binding to `0.0.0.0`\n\nmakes it reachable. Then map the port when you run it:\n\n```\ndocker run -p 8000:8000 -v $(pwd)/models:/app/models my-model\n```\n\n`-p 8000:8000`\n\nconnects port 8000 on your machine to 8000 in the container. Hit `http://localhost:8000/predict`\n\nand you're serving a model from a container.\n\nReal AI apps rarely run alone. You've got your model API, plus maybe a Redis cache for results and a vector database for embeddings. Starting three containers by hand, with the right flags and in the right order, gets old fast.\n\n**docker-compose** lets you define your whole stack in one `docker-compose.yml`\n\nfile:\n\n```\nservices:\n  model-api:\n    build: .\n    ports:\n      - \"8000:8000\"\n    volumes:\n      - ./models:/app/models\n    depends_on:\n      - redis\n\n  redis:\n    image: redis:7-alpine\n    ports:\n      - \"6379:6379\"\n\n  vector-db:\n    image: qdrant/qdrant:latest\n    ports:\n      - \"6333:6333\"\n    volumes:\n      - ./qdrant_data:/qdrant/storage\n```\n\nThen the entire stack starts with one command:\n\n```\ndocker compose up\n```\n\nOne command, three services, wired together and talking to each other. And because services can reach each other by name, your API connects to Redis at the host `redis:6379`\n\n, no IP addresses to chase down. Shut it all down with `docker compose down`\n\n. This is the moment most people fall in love with Docker.\n\nA short list of things worth doing from day one:\n\n`.dockerignore`\n\nfile.`.gitignore`\n\n, it keeps junk out of your image. At minimum: `__pycache__`\n\n, `.git`\n\n, `venv`\n\n, `*.pt`\n\n, and `data/`\n\n. Without it, you'll accidentally copy gigabytes of cache and datasets into your build.`-slim`\n\nor official ML base images.`python:3.11-slim`\n\nover the full image saves hundreds of megabytes. For GPU work, start from an official CUDA-enabled base like `pytorch/pytorch`\n\nso the driver stack is handled for you.`-e MY_KEY=...`\n\nor an `.env`\n\nfile), never hardcoded into the Dockerfile. Anyone with the image can read what's baked in.Go back to that teammate who couldn't run your model. With Docker, the entire conversation becomes:\n\n```\ngit clone your-repo\ndocker compose up\n```\n\nTwo commands. Same Python, same CUDA, same packages, same everything on their laptop, on the cloud GPU, on the production server. No \"what version are you on?\" No \"did you install ffmpeg?\" No 40-minute debugging session.\n\nYou don't need to master Kubernetes or become a DevOps engineer to get this. You just need a `Dockerfile`\n\n, a pinned `requirements.txt`\n\n, and maybe a `docker-compose.yml`\n\n. Start with the PyTorch example above, get one model running in a container today, and build from there.\n\nThe next time someone asks if your project works on their machine, you'll already know the answer.\n\nIt works on *every* machine.\n\n*Found this useful? Drop a comment with the trickiest \"works on my machine\" bug you've hit*", "url": "https://wpnews.pro/news/stop-saying-it-works-on-my-machine-docker-for-ai-engineers", "canonical_source": "https://dev.to/sachinsingh2156/stop-saying-it-works-on-my-machine-docker-for-ai-engineers-481g", "published_at": "2026-06-19 04:00:19+00:00", "updated_at": "2026-06-19 04:30:02.089502+00:00", "lang": "en", "topics": ["developer-tools", "machine-learning", "artificial-intelligence"], "entities": ["Docker", "PyTorch", "CUDA", "Python", "transformers", "FastAPI", "Uvicorn", "numpy"], "alternates": {"html": "https://wpnews.pro/news/stop-saying-it-works-on-my-machine-docker-for-ai-engineers", "markdown": "https://wpnews.pro/news/stop-saying-it-works-on-my-machine-docker-for-ai-engineers.md", "text": "https://wpnews.pro/news/stop-saying-it-works-on-my-machine-docker-for-ai-engineers.txt", "jsonld": "https://wpnews.pro/news/stop-saying-it-works-on-my-machine-docker-for-ai-engineers.jsonld"}}