# Build a Local LLM Chatbot with Ollama and Python

> Source: <https://dev.to/qingluan/build-a-local-llm-chatbot-with-ollama-and-python-mkd>
> Published: 2026-07-14 15:32:08+00:00

Imagine typing a question into your chatbot and getting a response in milliseconds, completely offline, with zero data leaving your machine. No API keys, no monthly subscription fees, and no privacy concerns about your data being sent to a cloud server. This isn’t a futuristic dream—it’s the reality of running a Local Large Language Model (LLM) on your own computer. With the rise of tools like **Ollama**, building a private AI chatbot in Python has become as simple as installing a few packages and writing a short script. Let’s dive in and build one together.

Before we write any code, it’s worth understanding why running an LLM locally is a game-changer. Cloud-based AI services like OpenAI or Anthropic are powerful, but they come with trade-offs: you pay per token, your data is processed on their servers, and you’re dependent on their uptime. A local LLM flips this model. You download the model once, run it on your hardware, and you have full control.

**Ollama** is the engine that makes this accessible. It’s a lightweight, open-source tool that simplifies running LLMs like Llama 3, Phi 3, or Mistral on macOS, Linux, and Windows. It handles model downloads, memory management, and inference, exposing a simple API that Python can easily interact with [1][2].

The first step is getting Ollama on your machine. Visit [ollama.com](https://ollama.com), click **Download**, and install the version for your operating system [2]. Once installed, verify it’s working by opening your terminal or Command Prompt and running:

```
ollama --version
```

If you see a version number, you’re ready to go. Next, you need a model. Ollama supports dozens of open-source models, but for a beginner-friendly chatbot, **Llama 3.2** is a great choice. It’s small, fast, and surprisingly capable.

To download it, run:

```
ollama pull llama3.2
```

This command fetches the model and stores it locally. Depending on your internet speed, this might take a few minutes [2][7]. Once it’s done, you can test it directly in the terminal:

```
ollama run llama3.2
```

Type a question like “What’s the capital of France?” and see the model respond. If you get a reply, Ollama is working perfectly.

Now let’s build the Python side. First, create a project folder:

```
mkdir local-llm-chatbot
cd local-llm-chatbot
```

Inside this folder, create a **virtual environment** to manage your dependencies cleanly. On macOS or Linux:

```
python3 -m venv chatbot
source chatbot/bin/activate
```

On Windows (Command Prompt):

```
python3 -m venv chatbot
.\chatbot\Scripts\activate.bat
```

On Windows (PowerShell):

```
.\chatbot\Scripts\Activate.ps1
```

Once the environment is active, install the necessary Python packages:

```
pip install langchain langchain-ollama ollama
```

We’re using **LangChain** and **langchain-ollama** because they provide a clean, high-level interface for interacting with Ollama models, making our code shorter and more maintainable [2][3][7].

Open your code editor (VS Code is a great choice) and create a file called `main.py`

. Here’s a complete, working Python script that creates a simple chatbot with conversation history:

``` python
from langchain_ollama import OllamaLLM

# Initialize the model
model = OllamaLLM(model="llama3.2")

# Conversation history to maintain context
history = []

print("🤖 Local LLM Chatbot (powered by Ollama + Llama 3.2)")
print("Type 'exit' to quit.\n")

while True:
    # Get user input
    user_input = input("You: ").strip()
    if user_input.lower() == "exit":
        print("Chatbot: Goodbye!")
        break

    # Add user message to history
    history.append({"role": "user", "content": user_input})

    # Generate response
    response = model.invoke(history)

    # Print and store response
    print(f"Chatbot: {response}")
    history.append({"role": "assistant", "content": response})

    # Optional: limit history to last 10 messages to save memory
    if len(history) > 20:
        history = history[-10:]
```

Save the file and run it:

```
python main.py
```

You’ll see a chat interface where you can type questions and get responses from Llama 3.2, all running locally. The `history`

list ensures the model remembers previous messages, giving you a more natural conversation experience [1][2].

This is just the foundation. Here are a few ways to make it even better:

`phi3`

, `mistral`

, or `gemma`

by pulling them with `ollama pull phi3`

and updating the `model`

parameter in the code.`ollama pull llama3.2`

and that Ollama is running (`ollama serve`

).`phi3`

) are faster but less capable. If you have a GPU, Ollama will automatically use it for faster inference.You now have a fully functional, local LLM chatbot running on your machine. No API keys, no subscriptions, no data leaks. Just pure, private AI.

The best part? This is just the beginning. You can extend this chatbot to read your documents, automate tasks, or even integrate it into your existing apps. The local AI revolution is happening right now, and you’re part of it.

**Try it out today**: Install Ollama, pull Llama 3.2, run the script above, and start chatting with your own AI. Share your results on Dev.to, tag me, and let’s build the future of private AI together. 🚀

*If you found this helpful, consider buying me a coffee ☕ — it keeps these articles coming!*

*Also check out my AI tools collection: AI 次元世界 — free AI tools for developers.*

*💡 Related: **Content Creator Ultimate Bundle (Save 33%)** — $29.99*