Live Transformer Graphs: Running and Editing LLM Architectures on a Custom CUDA Engine

A developer has built live transformer graphs into Leviathan, a custom CUDA engine that allows LLM architectures to be run and edited while the model is executing, with every part of the KV cache persisted as state that carries forward indefinitely. The project was presented as a Show/tell submission focused on manipulating a model during live operation.

Has anyone ever wondered what it would be like to edit the model while its actually running live? well, this is the result… also note every part of the Kvcache persisted as state carries forever forward. This project is built into Leviathan which I have previously mentioned in Show/tell it’s all about manipulation of when a model is running live for the most part.