Ollama: Confusion about model size

A developer criticizes Ollama, a tool for running large language models locally, stating that its model size display ignores context and only accounts for weights, and recommends using llama.cpp directly instead due to Ollama's non-standard model hosting and patched version of llama.cpp with suboptimal flags.

Yes, it’s not taking the context into account, just the weights alone. You can check it out at huggingface: Other than that, I’d recommend against using ollama. It does not use huggingface directly and has its own weird model hosting, and it uses a version of llama.cpp with some weird patches of theirs and not optimal flags. If possible, I’d suggest you just to use llama.cpp directly.