Local AI is an important topic for companies that work with sensitive data and want greater privacy and control over their information. This article explains how to connect Typemill to Ollama so that your AI workflows can run within your own infrastructure.
What you get #
- AI-assisted writing and help features in Typemill Core.
- Local AI processing without sending prompts and content to external AI providers.
- Support for additional AI-powered workflows in Typemill.
What you need #
- Typemill Core.
- No additional Typemill license required.
- Hardware with sufficient RAM and, ideally, GPU VRAM.
- Ollama and a locally running model.
Data Privacy and Data Control #
AI is becoming an integral part of many workflows, especially in software development, documentation, knowledge management, and content creation. Cloud-based AI services such as Claude and ChatGPT make powerful models easily accessible, but using them usually means sending your prompts and potentially sensitive data to external providers.
There are ways to reduce these risks, including enterprise agreements, data processing agreements, and privacy-oriented AI services. However, if you work with confidential documents, internal knowledge, or sensitive customer data, you may want to keep everything under your own control.
Local AI and Ollama #
Local AI offers a way to do exactly that. Instead of sending your data to an external AI provider, you run AI models on your own hardware. Your prompts, documents, and generated content can remain within your own infrastructure.
Local models are generally less capable than the most powerful frontier models, but they are already good enough for many everyday tasks, including writing assistance, translation, summarization, question answering, and working with documentation.
All you need is suitable hardware, a local model, and software that makes the model available to other applications. Tools such as Ollama and LM Studio make it relatively easy to download and run models on your own computer or server. For good results, you will need sufficient computing resources, including GPU VRAM or unified memory on supported hardware, such as Apple Silicon Macs.
Step 1: Install Ollama #
Ollama makes it easy to run large language models locally. It handles model downloads, model execution, and an API that other applications can use to communicate with your models.
Download and install Ollama for your operating system by following the instructions on the official Ollama website. On Linux, you can install it with the following command:
curl -fsSL https://ollama.com/install.sh | sh
Next, visit the Ollama model library to choose a model. Make sure you select a model that runs locally rather than a cloud model. Model pages provide details about available sizes and parameter configurations.
Download a model using the following command:
ollama pull <model-name>
Replace <model-name> with the model you want to use. To start a downloaded model directly from the terminal, use:
ollama run <model-name>
The model will load, and you can enter prompts directly in your terminal.
Test a few models if necessary. The best choice is not necessarily the largest or most powerful model, but the one that delivers good results for your tasks within your available hardware resources. Remember that the context window also requires memory, so a model that nearly fills your available VRAM may not leave enough room for the context and other processing requirements.
Step 2: Use the Ollama Endpoint #
Ollama runs a local service that applications can use to send prompts and receive responses. Its OpenAI-compatible API is available at the following address by default:
http://localhost:11434/v1
You can use this API to interact with your models directly or connect applications such as Typemill. By default, the service is only accessible from your local machine, not directly over the internet.
Step 3: Configure Ollama in Typemill #
To use Ollama with Typemill, open the system settings and navigate to the AI tab. Enter the following configuration:
- AI Adapter : OpenAI Chat Completions API
- API Base URL :
http://localhost:11434/v1 - Model : Enter the name of your downloaded model.
You can leave the other fields empty for now and save the settings.
Important: The address localhost always refers to the machine on which the application making the request is running. If Typemill runs inside a Docker container or on a separate server, you may need to configure a different address so that Typemill can reach Ollama.
Step 4: Use AI Features in Typemill #
Once Ollama is connected, you will see two new icons at the top of each article:
- Question mark: Opens AI-assisted help for Typemill.
- Magic wand: Opens the current article in Typemill's AI editor.
Both icons open the Kixote interface, an overlay for administration and AI assistance. You can also open the info tab in Kixote to view the available models. Select another model to override the default model configuration for your current workflow.
Test the help or generate feature to verify that the connection works and returns a valid response. The screenshot below shows the AI editor's Generate tab and the result of the predefined Proofread prompt applied to an article.
Typemill also offers other AI-assisted features, including:
- Auto-translations: If you use a multilingual Typemill setup, you will find an auto-translation button in the language tab of each article. Read more in thedocumentation .
- Ask the Docs: Use the Ask the Docs plugin to add a chatbot that answers questions based on your Typemill content. Read more inthis article .
You can also use AI to support specialized workflows such as Docs as Code and Tasks as Code. Additional AI features, including an MCP plugin, are in development.
Step 5: Make Ollama Available on Your Network #
So far, Ollama is only available on your local machine. You can also run it on a dedicated machine and make it available to other devices on your local network. This allows your team to share AI infrastructure and connect multiple applications, including Typemill, to the same models. This can be especially useful when using Typemill as an internal knowledge base.
For smaller setups, a single machine running Ollama may be sufficient. Larger teams and production environments require more careful planning around authentication, access control, and network security. If you expose an AI service to the internet, you must protect its API against unauthorized access and abuse.
In larger deployments, additional challenges arise. Multiple users may submit requests simultaneously, models require substantial memory and computing resources, and response times can vary depending on the workload. Reliability, monitoring, scalability, and resource management also become important.
If you plan to use local AI as a shared service across your organization, consult specialists in AI infrastructure and deployment. They can help you choose the appropriate hardware, model-serving architecture, security measures, and capacity planning strategy for your specific requirements.