Installing GPT4All, an Open-Source Chatbot Application for Running LLMs GPT4All, an open-source desktop application for running large language models locally, supports macOS, Windows, and Linux. The application enables users to download and run GGUF models from sources like Hugging Face, chat with local documents, and set up an API server behind a reverse proxy. Installation guides cover Ubuntu, Windows, and Mac with M-series processors. GPT4All is an open-source graphical desktop application for running large language models LLMs locally. It supports most desktop operating systems — macOS, Windows, and Linux — so you can run LLMs without an active internet connection, and it supports GGUF format models from sources including Hugging Face while also integrating with API-based providers like Groq, OpenAI, and Mistral. This guide covers installing GPT4All, downloading and running models, chatting with local documents, and enabling the built-in API server behind a reverse proxy. By the end, you'll have GPT4All installed with a local model running, a documents-aware RAG chat set up, and a remotely reachable API server secured with HTTPS. gpt4all.example.com domain for demonstration purposes.You can install GPT4All on Windows, macOS, or Linux. GPT4All is available only for Ubuntu and Debian-based desktop workstations. You can install it using the latest installation binary or Flatpak. The steps below download and install the latest installation binary on an Ubuntu 24.04 desktop workstation. 1. Log in to your Ubuntu desktop workstation using VNC, NoMachine, or a remote desktop client, then open a new terminal or press CTRL + ALT + T . 2. Download the latest GPT4All .run installation binary for Linux from the official website: bash $ wget https://gpt4all.io/installers/gpt4all-installer-linux.run 3. Enable execute permissions on the downloaded file: bash $ chmod +x gpt4all-installer-linux.run 4. Execute the file to open the GPT4All installation wizard you need a full desktop environment to run the application : $ ./gpt4all-installer-linux.run 5. Click Next , verify the installation path, select all components, accept the license, and click Install to complete the wizard, then click Finish to close the installer. 6. List the GPT4All installation directory and verify all application files are available: bash $ ls ~/gpt4all Output: bin InstallationLog.txt Licenses network.xml share components.xml installer.dat maintenancetool plugins gpt4all-32.png installerResources maintenancetool.dat qml gpt4all-48.png lib maintenancetool.ini resources 7. List your Desktop path and verify a GPT4All.desktop file is available: bash $ ls ~/Desktop Output: GPT4All.desktop 8. Create an applications directory for your user: bash $ mkdir -p ~/.local/share/applications 9. Move the GPT4All.desktop file into it: bash $ mv ~/Desktop/GPT4All.desktop ~/.local/share/applications/ 10. Open your applications menu, search for GPT4All, and click the icon to launch the chat interface. GPT4All supports x86 and ARM-based processors for Windows, with a single installer supporting CPU or GPU builds. 1. Download the latest GPT4All .exe installation binary from the official website. 2. Open the downloaded .exe file to launch the installer, click Next, verify the installation path, select the components to install, accept the license, keep the default start menu shortcut, and click Install. 3. Monitor the installation process, click Finish to close the installer, then open the Windows start menu, search for GPT4All, and open it from the list of installed applications. Follow these steps to install GPT4All on a Mac with an M-series processor. 1. Download the latest GPT4All .dmg installation binary from the official website. 2. Open the downloaded installer package and double-click the gpt4all-installer-darwin file to open the installation wizard. 3. Click Next on the Installation Folder page you can keep the default path , click Next again to review the single component gpt4all , accept the License Agreement, and click Install. 4. Monitor the installation process, click Finish, then open the Launchpad and search for GPT4All to launch the chat interface. GPT4All does not include any models by default. You download models from the built-in GPT4All repository or external sources such as Hugging Face, including distilled, instruct, reasoning, uncensored, and censored variants. 1. Open GPT4All from your applications menu and click Models on the main navigation menu. 2. Click Add Model to explore the Explore Models page, which offers: 3. Scroll or search for a specific model, such as Llama3.2 1B Instruct, and verify its requirements: 4. Click Download, then click Models on the navigation menu and verify the model is downloaded and available. Downloaded models are available on the Chats tab. All chats are private and local to your workstation. 1. Click Chats on the main navigation menu, click Select a Model, and choose a model. 2. Enter a prompt in the Send a message field and press Enter to start the chat. 3. Monitor the token count and model response rate, and verify the response matches your prompt. 4. Enter another prompt to continue chatting with context from the original prompt, and use Delete, Edit, or New Chat in the history pane to manage conversations. GPT4All supports Retrieval-Augmented Generation RAG by default through the LocalDocs menu option, letting you upload documents and chat with an LLM using them to improve response quality. 1. Click Settings, select LocalDocs, and verify the list of supported file extensions. 2. Click the Embeddings Device option and select CUDA if available to use your GPU for embedding local documents. 3. Click LocalDocs on the main navigation menu, then click Add Doc Collection to create a new collection. 4. Enter a collection name, click Browse to select the path containing your documents, and click Create Collection to scan them. 5. Verify the embeddings count and the model in use nomic-embed-text-v1.5 by default , and monitor progress for each document. 6. Click Chats, start a new chat, and click LocalDocs in the top-right to select your documents collection for RAG. GGUF GPT-Generated Unified Format stores, loads, and infers large language models efficiently, optimized for memory usage and fast loading. You can download .gguf models from platforms like Hugging Face and manually add them to GPT4All. 1. Open a web browser and visit Hugging Face to search for a model — for example, Mistral GGUF. 2. Select a model from the list, such as mistralai/Devstral-Small-2507 gguf, click Files and Versions, and select a quantization version — for instance, Devstral-Small-2507-Q4 K M.gguf — noting its file size. 3. Open a new terminal window and check available disk space: bash $ df -h 4. Download the file from your browser, then navigate to your Downloads folder in the terminal: bash $ cd ~/Downloads 5. Move the downloaded .gguf file to the GPT4All models directory: bash $ mv Devstral-Small-2507-Q4 K M.gguf ~/.local/share/nomic.ai/GPT4All/ 6. List the directory and verify the available .gguf model files: bash $ ls ~/.local/share/nomic.ai/GPT4All/ Output: Devstral-Small-2507-Q4 K M.gguf Llama-3.2-1B-Instruct-Q4 0.gguf localdocs v3.db test write.txt 7. Close and reopen GPT4All to load the newly added model, then navigate to the Models tab to verify it's available. 8. Navigate to Chats, click New Chat, select the new model, and enter a prompt such as write a short story about a robot learning to code, with a focus on arrays, then press Enter to generate a response. GPT4All includes a built-in API server for interacting with models over REST endpoints, letting you integrate GPT4All into existing applications. 1. Click Settings, navigate to the Advanced section, and click Enable Local API Server. 2. Note the default API Server Port, then open a new terminal window and send a GET request to list all available models: bash $ curl http://localhost:4891/v1/models 3. Send a GET request for a specific model's details replace Devstral-Small-2507-Q4 K M.gguf with a model available on your server : bash $ curl http://localhost:4891/v1/models/Devstral-Small-2507-Q4 K M.gguf 4. Send a POST request to generate a text completion: bash $ curl -X POST http://localhost:4891/v1/completions \ -H "Content-Type: application/json" \ -d '{ "model": "Devstral-Small-2507-Q4 K M.gguf", "prompt": "What is the sum of 8, 5, and negative 200?", "max tokens": 500 }' The API returns a JSON response with the completion and a few other details. GPT4All listens for API requests on the 127.0.0.1 localhost address, which limits connections to your local system. Installing Nginx as a reverse proxy enables secure network connections to the API server by forwarding requests to the localhost address. 1. Update the APT package index: bash $ sudo apt update 2. Install Nginx: bash $ sudo apt install nginx -y 3. Create a new virtual host configuration file: bash $ sudo nano /etc/nginx/sites-available/gpt4all.conf 4. Add the following configuration, replacing gpt4all.example.com with your domain name: server { listen 80; server name gpt4all.example.com; location / { proxy pass http://127.0.0.1:4891; proxy http version 1.1; proxy set header Upgrade $http upgrade; proxy set header Connection 'upgrade'; proxy set header Host $host; proxy cache bypass $http upgrade; } } Save the file and exit. This forwards all connection requests from the gpt4all.example.com domain to the GPT4All API server on port 4891 . 5. Enable the site by linking it to sites-enabled: bash $ sudo ln -s /etc/nginx/sites-available/gpt4all.conf /etc/nginx/sites-enabled/gpt4all.conf 6. Remove the default Nginx virtual host configuration: bash $ sudo rm /etc/nginx/sites-enabled/default 7. Test the Nginx configuration for syntax errors: bash $ sudo nginx -t Output: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: configuration file /etc/nginx/nginx.conf test is successful 8. Restart Nginx to apply the changes: bash $ sudo systemctl restart nginx 9. Allow network connections to Nginx through the firewall and reload it: bash $ sudo ufw allow "Nginx Full" $ sudo ufw reload 10. Install Certbot to generate a Let's Encrypt certificate: bash $ sudo apt install -y certbot python3-certbot-nginx 11. Generate a certificate for your domain replace admin@example.com with your email address : bash $ sudo certbot --nginx -d gpt4all.example.com -m admin@example.com --agree-tos 12. Restart Nginx to apply the certificate changes: bash $ sudo systemctl restart nginx With the reverse proxy in place, GPT4All accepts remote API requests forwarded to the localhost API server port. 1. Send a GET request to list all available models using your domain: bash $ curl https://gpt4all.example.com/v1/models 2. Send a GET request for a specific model's details: bash $ curl https://gpt4all.example.com/v1/models/Qwen2-1.5B-Instruct chat binary in C:\Users\