# Does an LSP help a coding agent? > Source: > Published: 2026-09-08 20:50:46+00:00 It was a busy weekend with all the activity coming out of OpenAI. If you have been living under a rock, they introduced a new class of model, [GPT 6 - Astra](https://openai.com/index/gpt-6-astra/). I've been playing around with it somewhat. That is not what this article is about, though. I do not have an unlimited budget to run lots of evals against a model like this, so any opinion I have on it would be purely anecdotal. There are already plenty of anecdotal opinions to read about all over the internet. Today's article is about a question I was wrestling with while building out my Pi agent. I was using my agent as a pair programmer and realised that, with Pi, the agent did not have access to the Language Server Protocol (LSP). If you are not familiar with LSPs, they tend to be services on your machine that IDEs use to analyse files with recognised code extensions. For example, if I am in a file with a `.rs` extension and have a line that would cause a compiler error in the Rust compiler, and my IDE has the Rust LSP installed, the IDE can display the error without me having to run the compiler. In a recent coding session, I realised that the agent was having to run `cargo check` to review my compiler errors. That made me realise the lack of capability. I started looking up Pi LSP extensions and came across [this one](https://github.com/narumiruna/pi-extensions/tree/main/packages/pi-lsp). If you are looking for an LSP for Pi, it seems to do the job just fine. Credit to the author, narumiruna. The README contains a link to a comment on the OpenAI Codex repository where an OpenAI developer questioned what additional benefits introducing an LSP in `AGENTS.md` would bring beyond asking an agent to run a linter or type checker. It was not just any engineer, either. The OpenAI engineer is the author of Pyright, a popular Python LSP: Eric Traut. I thought this was an interesting debate, so I decided to see whether using an LSP extension improved the performance of my Pi agents. For this task, I wanted to make an enhancement to the eval harness. I needed to introduce a capability profile for each agent when configuring an evaluation run. Originally, to test different extensions or capabilities in a Pi agent, you had to either modify the Docker image or write a specific section in each evaluation. This meant having copies of each evaluation for every capability profile. Because the evaluation harness also used the Agent Shell adapter to configure agents, I needed to make this change so packages and extensions for the Pi harness could be managed through Agent Shell. [Version v0.4.0](https://github.com/ScottRBK/agent-shell/releases/tag/v0.4.0) introduced this. It currently only supports the Pi harness, but I plan to extend package management to the other harness types in Agent Shell as well. With Agent Shell's package and extension management in place, I added a feature to the eval-harness that added a capability profile. For Pi agents, this means you can compare an agent with one set of capabilities against an agent with another. For example, you can test a Pi agent with and without an LSP using the same model. After making the changes, I put together an evaluation run. Given the stochastic nature of large language models, it was important to run each evaluation three times. I used eight evaluations across three models: `mimo-v2.5` and `muse-spark-1.3-contributor`, both provided by opencode-go, and OpenAI's Luna. I compared Pi with no added package against Pi with `@narumitw/pi-lsp@0.49.7`. I set the reasoning effort to medium for all models. Muse Spark came out on top overall. Its base profile scored 83.0%, compared with 81.3% with LSP. Luna scored 75.3% without LSP and 68.1% with it. MiMo was the only model where LSP improved the score, moving from 65.6% to 67.5%. Across the three model pairs, the LSP versions used 124,737 fewer tokens in total, a 6.4% reduction. MiMo used 7.4% fewer tokens, Muse Spark 1.7% fewer, and Luna 10.7% fewer. The LSP versions took slightly longer overall, although Luna was faster with LSP. I would describe these results as inconclusive. From my perspective, that aligns with what Eric was suggesting in his comment. Personally, I will probably leave the LSP extension off, except perhaps when I am pair programming with the LLM. It is just some additional tools and context, and I cannot see an obvious benefit right now. I would be interested to hear what you think in the comments or on Discord. I hope this was informative, interesting, or inspiring, and got you thinking about how you can shape your agentic harnesses, whatever your setup may be.