Not so long ago I wrote a guide on how to get GitHub Copilot running with a local model in Visual Studio Code. Since then Copilot subscriptions have got much more expensive, local models have got much more powerful and getting local Copilot up and running has got much easier. Seems like it's time for an updated guide, so here we go.
-
A computer, relatively modern, with at least 16GB of RAM and 12GB of VRAM, if it's a PC or 16GB of RAM if it's a Mac.
-
An operating system: MacOS, Windows or pretty much any Linux
-
Visual Studio Code > 1.122
-
Enough bandwidth and/or patience to download a 7GB model
-
A GitHub Copilot subscription
-
A GitHub account
-
A $3000 graphics card, although if you've got one it won't hurt
We'll need a server to host our local model, Lemonade gives us a cross-platform install for that, you can use your favourite package manager or grab an installer from their install page. I'm on MacOS today so I'll:
Now the server's installed we can go ahead and load a model. I've chosen Granite 4 tiny from IBM, not because it's the smartest model in the room but because it has acceptable performance on our minimum hardware requirements even with 128k of context and supports tool calling. If you've got more powerful hardware there are much better options available but for now we'll start with something we know works.
The model is about 7GB so it'll take a minute to download, but once it's successfully pulled we can load it.
And now we've got a local model up and running, time to hook it up.
Since version 1.121 VS Code has supported Custom Endpoint providers for agents and chat completions, and since version 1.122 that functionality no longer requires a Copilot license or a GitHub login.
We'll start by searching for and selecting Manage Language Models in VS Code settings.
Once the list of models pops up we'll click + Add Models and we'll see a handy Custom Endpoint option.
Let's select that option and when it asks for a name call our group Lemonade, leave the API Key blank (because we haven't set one in Lemonade) and select Responses Responses API as the API Type.
And now VS Code will drop us into the chatLanguageModels.json where we'll populate the id of our lemonade model, granite-4.0-h-tiny-GGUF-Q8_0, the name we want it to show up with, granite and the url of our Lemonade API http://localhost:13305/v1/responses. You can also see that I've set toolCalling to true so the model can use tools and vision to false because Granite doesn't support images as input.
Once we save this file then our model becomes available in Copilot chat.
And now we can say Hello to our entirely local Copilot 👋
So what next? Well it's going to become clear pretty quickly that Granite is very limited in it's role as a coding agent/assistant, but that doesn't mean it's entirely useless. You can put it to work writing unit tests or summarising code in the background, think of it as an intern assistant who, but of course don't trust it's code.
Once you're done trying out Granite I'd suggest moving on to see what other models your hardware can run and how fast. That's a pretty huge topic so I'll just recommend a few that I'd suggest you try in order.
Good luck and please leave a comment if you have other models or configs to recommend.
Missing agentic features #
If you can chat to the model but don't see agentic features available, for example being able ask the agent to edit code for you, then make sure that chat.agent.enabled is turned on.
No utility model #
If you get the error No utility model is configured for 'copilot-utility-small' while the selected main agent model is BYOK. it's because of this Copilot issue. Long story short Copilot doesn't know what to use as it's Utility Model, you can fix it by setting the value in preferences like this: