How to download a model and run it with Ollama locally?

A user on the Hugging Face forum asked how to download a model from Hugging Face and run it locally with Ollama on a MacBook Pro. Other users recommended tools like LMStudio and explained that Ollama wraps llama.cpp, supporting any Hugging Face model in GGUF format. Steps were provided to convert GGUF models into Ollama format, though users noted challenges with embedding models.

I have just installed Ollama on my Macbook pro, now how to download a model form hugging face and run it locally at my mac ? Thank you for your response, is there any other recommend tool to run models locally than Ollama? Why should I use Ollama? Am I on the right track? Certainly Here are some alternative tools for running models locally: - PyTorch - TensorFlow - ONNX Runtime - OpenVINO - TensorFlow Lite Each tool offers unique features and advantages, so you can choose based on your specific requirements. Thank you for the information, I really appreciate it Can’t find it… Any other options? You can use git to clone the model to your local storage. As long as you have git installed on your computer just use the “git clone” command followed by the URL of the model on hugginface. For example: git clone mistralai/Mistral-7B-Instruct-v0.2 · Hugging Face https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2 pistha https://discuss.huggingface.co/u/pistha 8 So the downloaded model will be in quantized form. Right? Can we use this to fine tune ? I dont see any config.json file. Any idea ? @hopewise /u/hopewise You do realize that both of @KeelyPowers /u/keelypowers 's answers were generated by a LLM… right? No But I am not shocked sense this is huggingface forum My I ask,after my downloading models, how can i use ollama to load it? My downlaoding is in transformer format,with files like config.json,model-00001-of-00002.safetensors , pytorch model-00003-of-00003.bin special tokens map.json tokenizer.model No modelfile,how can I load it with ollama Hi, Ollama is a wrapper on top of Llama cpp: GitHub - ggerganov/llama.cpp: LLM inference in C/C++ https://github.com/ggerganov/llama.cpp , which supports any HF model which can be run from the terminal. I would recommend checking it out, along with LMStudio https://lmstudio.ai/ , which provides a nice UI on top of it, it supports any HF model in the GGUF format. qdrddr https://discuss.huggingface.co/u/qdrddr 13 Here are the steps to convert GGUF models from HF into the Ollama format qdrddr https://discuss.huggingface.co/u/qdrddr 14 Though I have trouble understanding how to do that for the embeddings model. How can I import Huggingface GGUF Embeddings model to Ollama format if the model card page does not include the Modelfile? For example, for this embeddings model: dranger003/SFR-Embedding-Mistral-GGUF · Hugging Face https://huggingface.co/dranger003/SFR-Embedding-Mistral-GGUF Or this one: GritLM/GritLM-8x7B · Hugging Face https://huggingface.co/GritLM/GritLM-8x7B qdrddr https://discuss.huggingface.co/u/qdrddr 15 I read these articles how to convert a model to Ollama format, but they are not clear on the embeddings Is there a difference how I supposed to create the Model file for Embeddings model vs. the LLM model? Where can I get expected for the model Parameters and Template values for a given model if they are not provided? To download and run a model with Ollama locally, follow these steps: Install Ollama : Ensure you have the Ollama framework installed on your machine. Download the Model : Use Ollama’s command-line interface to download the desired model, for example: ollama pull <model-name . Run the Model : Execute the model with the command: ollama run <model-name . Configure Settings : Adjust any necessary settings or parameters as required by your specific use case. Refer to Ollama’s documentation for detailed instructions and additional options. zele https://discuss.huggingface.co/u/zele 17 This guy has the exact video you’re looking for. The first half is about Quantized/GGUF models. But around minute mark 4:23 he gets into regular models, like what you’re asking about MrAR https://discuss.huggingface.co/u/MrAR 18 What about other architectures? In Ollama, models supported by Llama.cpp are basically usable, so you can use almost any architecture model in the same way. One thing to note about recent models is that Qwen 3 seems to perform worse with quantization smaller than Q6, so be careful. Hi. It’s 2026 now, and a lot easier than anything in this thread. Step 1: Go to the App Store and download a local LLM app, such as Locally AI, Private LLM, Liquid Apollo, Solair AI, PocketPal AI, Enclave, MITHRIL, fullmoon, etc. Step 2: Open the Model menu in the app and download the model s you’d like to use. Most apps don’t show models that aren’t compatible with your device, but some do, so keep an eye on that. Step 3: Tweak your LLM parameters. Step 4: Enjoy That’s it. Nice and simple. From the hugging face repo url I did with sonnet-5 model . Exists a button “Use this model” → “Ollama” Thanks hugging face for do it so easy Once you click it you will some something like this: ollama run hf.co/oroboros-labs/claude-sonnet-5