How to run LFM2.5-24B-A2B Q8_0 on 2× RTX 5060 Ti 16 GB

A benchmark of LFM2.5-24B-A2B Q8_0 running on 2× RTX 5060 Ti 16 GB GPUs found a 0/3 delivered-pipeline pass rate across three v5 runs on the 32 GB tensor-split tier, each capped at 130,000 tokens, despite all three runs finishing cleanly with review, module and tests. The same setup passed 30 of 30 needle-in-a-haystack probes, including at 130,864 prompt tokens, with prefill decaying from 8,070 to 5,453 tok/s and decode from 149 to 104 tok/s as the context window filled. The author reports a 76-79% thinking tax and 64-71% eight-gram duplication in emitted characters and concludes the model should be kept as a long-context general driver, not routed Elixir work through it.

Warning: this is a temporary briefing, written by an LLM and not yet evaluated or rewritten by a human to get rid of the signature AI-slop writing style. It’s published as a draft despite its shortcomings, so that you can find out how to run it and what to expect in terms of pp and tg on this hardware, as well as what to expect in terms of coding performance on this benchmark, grosso modo . Three v5 runs on the 32 GB tensor-split tier, each capped at 130,000 tokens — near the full 131,072 served slot. The cap tracks the window on purpose: a short mid-answer cap can retire a model that would have finished, so a run is only comparable to the slot it was given. All three runs finished cleanly with review, module and tests — and none produced a runnable pipeline. Two modules do not compile; the third compiles while its test file references handles that were never bound. Delivered-pipeline pass rate: 0 / 3. Thinking tax ran 76-79 % of emitted characters, with 64-71 % eight-gram duplication across the whole answer — repetitive reasoning, never a terminal loop. The needle ladder is the other half of the story. One unique anchor, buried at every tenth of the served slot, three repeats per depth, each request opening with a fresh nonce so every step measures cold prefill. Thirty of thirty hit, including at 130,864 prompt tokens. Prefill decays smoothly from 8,070 to 5,453 tok/s and decode from 149 to 104 tok/s as the window fills — a soft curve, not a cliff. Methodology, disclosed: the first needle attempt used a 48-token answer cap and returned empty content at every depth — archived, not measured. The 90 % and 100 % steps initially failed with a context-overflow error because a byte-based token estimate undershot the real count; the runner now feeds the server’s reported token count back into the estimate, and the re-run converged within 0.02 % of the target depth. Representative output Run 1’s TtlStore — trimmed to the public API and eviction handler. The full module plus its ExUnit suite shipped fenced; this is the parts that actually run: python defmodule TtlStore do use GenServer def start link name do GenServer.start link MODULE , name, end def start link name, opts do default ttl = Keyword.get opts, :default ttl, 60 GenServer.start link MODULE , name, :default ttl = default ttl end def put pid, key, value do validate binary key validate binary value GenServer.call pid, {:put, key, value, nil} end def put pid, key, value, opts do validate binary key validate binary value GenServer.call pid, {:put, key, value, opts} end def get pid, key do GenServer.call pid, {:get, key} end def delete pid, key do GenServer.call pid, {:delete, key} end def stats pid do if is pid pid do GenServer.call pid, :stats else raise ArgumentError, "invalid pid" end end def handle info :evict, state do now = System.system time :second new store = Enum.filter state.store, fn { k, { v, expires at}} - expires at now end | Enum.into %{} new state = %{state | store: new store} Process.send after self , :evict, state.eviction interval {:noreply, new state} end end Route decision: keep it as a long-context general driver. Do not route Elixir work through it.