Every token costs money. Every API call adds up. And your data goes to their servers.
brew install ollama
ollama pull llama3.2
ollama serve
Now you have a GPT-4 level model running on your MacBook. Free. Private. Fast.
import requests
def ask_local_ai(question):
r = requests.post(
"http://localhost:11434/api/generate",
json={"model": "llama3.2", "prompt": question, "stream": False}
)
return r.json()["response"]
answer = ask_local_ai("Explain Docker in one paragraph")
print(answer)
| Model | Size | Best For |
|---|---|---|
| llama3.2 | 2GB | General use |
| codellama | 4GB | Code tasks |
| mistral | 4.4GB | Fast reasoning |
| phi3 | 2.2GB | Lightweight tasks |
| gemma2 | 5.4GB | Complex reasoning |
I built 45 tools using this stack. All free. All local. All open source.
github.com/amrendramishra/ai-tools
VP at JPMorgan Chase. Building AI tools at amrendranmishra.dev
source & further reading
dev.to — original article
Golden Hours
SolSpot: An Open Model Scores the Sky, So You Can Close the Screen 🌿
Five switches. Two app stores. How I built Fud AI’s release pipeline