{"slug": "running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it", "title": "Running Mistral offline: the tokenizer is on disk, but the library can't find it", "summary": "A developer identified and fixed a bug in mistral-common, Mistral AI's open-source request-preparation library, that caused offline tokenizer loading to fail with a FileNotFoundError when models were stored outside the default Hugging Face cache. The fix, merged by a Mistral AI maintainer in PR #349, makes the file-listing step honor the user-supplied cache_dir the same way hf_hub_download does, so air-gapped deployments with a dedicated model disk can load tokenizers without network access. The developer notes Transformers and vLLM likely inherit the same issue offline, though that was not tested end to end.", "body_md": "An inference server cut off from the Internet. A dedicated disk for models. Mistral's tokenizer downloaded in advance, sitting in its folder. And at startup:\n\n```\nFileNotFoundError: No local files found for the repo ID mistralai/Mistral-7B-v0.1 and revision None.\n```\n\nThe file is there. The library looks somewhere else.\n\nI found this bug in [mistral-common](https://github.com/mistralai/mistral-common), Mistral AI's open-source library that prepares requests for its models. The fix was reviewed, approved and merged by a Mistral AI maintainer ([PR #349](https://github.com/mistralai/mistral-common/pull/349), [issue #348](https://github.com/mistralai/mistral-common/issues/348)).\n\nTeams running Mistral models **without Internet access** that store their models **outside the default Hugging Face cache**. That is exactly the enterprise setup: air-gapped network for security, corporate proxy, a dedicated model disk on the GPU server.\n\nTwo situations trigger it:\n\n`local_files_only=True`);` HF_HUB_OFFLINE=1`) and the library falls back to local files on its own.\nWith the default cache, or with network access, nothing happens. That is why the bug stays invisible in development and shows up on deployment day.\n\nOffline, loading the tokenizer chains two functions:\n\n`list_local_hf_repo_files` lists the cached files to find the tokenizer (`tekken.json`). It `HF_HUB_CACHE`, and had no `cache_dir` parameter.`hf_hub_download` then fetches the file. It does receive \n\n```\n# before: the listing ignores the folder chosen by the user\nrepo_cache = Path(huggingface_hub.constants.HF_HUB_CACHE) / ...\n```\n\nIf the tokenizer only lives in `cache_dir`, the listing comes back empty and the function gives up before even trying to read the file. The reverse case fails too, one step later.\n\nBoth steps now read the same folder: `cache_dir` if given, otherwise the default cache, exactly the rule `hf_hub_download` follows.\n\n```\ncache_root = Path(cache_dir) if cache_dir is not None else Path(huggingface_hub.constants.HF_HUB_CACHE)\nrepo_cache = cache_root / huggingface_hub.constants.REPO_ID_SEPARATOR.join([\"models\", *repo_id.split(\"/\")])\n```\n\n`Path`, full offline load, fallback after a network error, The maintainer asked for a single change and applied it himself: removing a code comment. The code itself did not change.\n\nIf you are affected and cannot upgrade mistral-common yet: point `HF_HUB_CACHE` at the same folder as your `cache_dir`. Both steps then read the same place.\n\nFrom reading their source code, Transformers (`cache_dir`) and vLLM (`--download-dir` with `tokenizer_mode=mistral`) pass that folder down to mistral-common, so they are probably affected offline too. I did not test that end to end.\n\nDeploying Mistral models on an air-gapped network? Which other traps have you hit?\n\nLet's talk, or get your own agents audited: [LinkedIn](https://www.linkedin.com/in/zakariakhchiche/) · [Malt](https://www.malt.fr/profile/zakariakhchiche) · [Medium](https://medium.com/@ZKHCHICHE) · [Website](https://zakariakhchiche.github.io/)\n\nWant your team to build agents like these? I run a hands-on Copilot Studio training (in French, with Spar-x, Qualiopi-certified, eligible for OPCO funding in France): [https://zakariakhchiche.github.io/formation-copilot-studio/](https://zakariakhchiche.github.io/formation-copilot-studio/) and a free AI Act article 4 kit: [https://zakariakhchiche.github.io/kit-ai-act/](https://zakariakhchiche.github.io/kit-ai-act/)", "url": "https://wpnews.pro/news/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it", "canonical_source": "https://dev.to/zakaria_khchiche_490919ed/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-cant-find-it-5a2", "published_at": "2026-10-05 19:07:52+00:00", "updated_at": "2026-10-05 19:18:23.479947+00:00", "lang": "en", "topics": ["ai-tools", "large-language-models", "ai-infrastructure", "mlops", "developer-tools"], "entities": ["Mistral AI", "mistral-common", "Hugging Face", "Transformers", "vLLM", "Zakaria Khchiche"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it", "markdown": "https://wpnews.pro/news/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it.md", "text": "https://wpnews.pro/news/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it.txt", "jsonld": "https://wpnews.pro/news/running-mistral-offline-the-tokenizer-is-on-disk-but-the-library-can-t-find-it.jsonld"}}