Alan Yahya3 min read
Background #
A lot of LLM use revolves around agents. Unlike a simple chat completion, an agent will analyse your message, create a plan, and apply tools to achieve a task. LLMs are stateless, they only see an input and output, so an agent needs to be able to manage context and delegate tasks.
To improve these systems, we used to directly edit the LLM weights. This was a trade-off, as many of the emergent properties of an LLM come from their generality, which model fine tuning would inevitably degrade.
Overall, fine tuning would work well for problems that could be solved in one shot. However, most real-world tasks are solved by an agent in multiple steps. As a result, most approaches now focus on optimising the agent, rather than altering the LLM directly.
Working with agents #
As agent workflows mature, users will come to expect a common set of features, which make LLMs easier to work with. For agent-heavy workflows, these features quickly become essential.
This includes managing how knowledge is retained, including the ability to fork, rewind or export agent sessions, which can span hundreds of thousands of lines of text. It also includes managing how changes are applied, such as the ability to review suggestions in-line, alter suggestions using different context, or merge multiple changes logically.
Vertical applications #
Once a core agent workflow is established, each vertical can begin to differentiate itself. Firstly, in terms of how the persistent agent memory is managed. The information you capture, and the hierarchy of information, will depend on the field you are working in. For example in coding you might persist top-level functions. In law, you might persist common entities between sections and clauses.
Secondly, in terms of the design of the workspace itself. One example is Cursor, which was initially a plugin, and now provides a standalone desktop application. Another example is the terminal interface, like Claude Code, which is a very high level of abstraction. Users with decades of familiarity using a particular app (like MS Word) will naturally prefer the latter. Particularly in a document where you need to audit every sentence, most users are more comfortable doing so in Word over a terminal.
Still, it leaves us in an interesting situation. We have multi-agent multi-document applications within a tool designed for single document editing (MS Word). Most AI providers provide a web application, where they have more control over how the application works, but moving users off tools like Word will be a gradual process.
Consolidation #
It is increasingly common to glue multiple apps together using a MCP or similar connector. This works, but can be a clumsy solution, as custom connectors between systems not built for interoperability can surface problems and edge cases.
There are obvious advantages to software consolidation, that are compounded by the high volume nature of LLMs. In the legal technology space, we can see this with infrastructure tools acquiring document editors (IE Filevine acquiring Pincites or Relativity acquiring Gavel).
This fits into an overarching picture, one of software companies locked in an existential struggle. These companies are no longer content to own a single slice of the stack, and as a result we see many of them building out their own complete agent workspace.