You baked the model into the image. One env var can silently un-bake it

A developer explains how baking Hugging Face model weights into a serverless GPU image at build time can be silently undone by a runtime environment variable. If HF_HOME is overridden at runtime, the loader ignores the baked weights and falls back to downloading from the network, negating the cold-start optimization. The developer recommends attaching no volume, verifying at boot, and treating the cache path as part of the build contract.

On serverless GPU, cold start is image pull plus model load. The single biggest win is usually to stop downloading weights at boot: download them at build time so they ship inside the image and load from local disk instead. The setup is two lines. Point the cache at a path inside the image, then fetch during the build: python ENV HF HOME=/opt/huggingface RUN python -c "from huggingface hub import snapshot download; snapshot download '<repo ', revision='<pinned ' " snapshot download writes into HF HOME , that directory becomes an image layer, and at runtime the loader finds everything already on disk. Here's the part worth knowing. The loader resolves weights through HF HOME at runtime . So if anything in your runtime environment sets HF HOME to a different path — a network volume, a mount, a leftover template variable — the loader looks there, finds an empty cache, and quietly falls back to downloading from the network. Nothing errors. Your image still contains the weights, sitting in a directory nobody reads anymore. You just paid to bake them, and you're paying the download on every cold start anyway. The only symptom is that cold starts went back to what they were before you did the work, which is easy to blame on the platform. Attach no volume at all. If the worker has no volume, there's no tempting path to point the cache at. In my setup the endpoints run with volume size zero for exactly this reason: everything the worker needs is in the image, so a volume is not an optimization, it's a way to reintroduce the network. Verify at boot, not by feel. Watch a cold worker's log. If you see download progress for weights, the bake is being bypassed, full stop. A baked image reaching the network for weights is always a misconfiguration, never a normal path. Treat the cache path as part of the build contract. The Dockerfile that baked the weights and the runtime env that reads them have to agree. That agreement is invisible in both files individually, so write it down where whoever edits the template env will see it. Baking has real costs: the image gets large, builds take much longer, and updating the model means rebuilding and redistributing the whole thing. If your weights are small or your traffic keeps workers warm, a volume or a runtime fetch may genuinely be the better trade. This is about the case where you already chose to bake — because then paying the build cost and the download cost is the worst of both.