How to run LFM2.5-24B-A2B Q8_0 on 2× RTX 5060 Ti 16 GB

Warning: this is a temporary briefing, written by an LLM and not yet evaluated or rewritten by a human to get rid of the signature AI-slop writing style. It’s published as a draft despite its shortcomings, so that you can find out how to run it and what to expect in terms of pp and tg on this hardware, as well as what to expect in terms of coding performance on this benchmark, grosso modo.

Three v5 runs on the 32 GB tensor-split tier, each capped at 130,000 tokens — near the full 131,072 served slot. The cap tracks the window on purpose: a short mid-answer cap can retire a model that would have finished, so a run is only comparable to the slot it was given.

All three runs finished cleanly with review, module and tests — and none produced a runnable pipeline. Two modules do not compile; the third compiles while its test file references handles that were never bound. Delivered-pipeline pass rate: 0 / 3. Thinking tax ran 76-79 % of emitted characters, with 64-71 % eight-gram duplication across the whole answer — repetitive reasoning, never a terminal loop.

The needle ladder is the other half of the story. One unique anchor, buried at every tenth of the served slot, three repeats per depth, each request opening with a fresh nonce so every step measures cold prefill. Thirty of thirty hit, including at 130,864 prompt tokens. Prefill decays smoothly from 8,070 to 5,453 tok/s and decode from 149 to 104 tok/s as the window fills — a soft curve, not a cliff.

Methodology, disclosed: the first needle attempt used a 48-token answer cap and returned empty content at every depth — archived, not measured. The 90 % and 100 % steps initially failed with a context-overflow error because a byte-based token estimate undershot the real count; the runner now feeds the server’s reported token count back into the estimate, and the re-run converged within 0.02 % of the target depth.

Representative output #

Run 1’s TtlStore — trimmed to the public API and eviction handler. The full module plus its ExUnit suite shipped fenced; this is the parts that actually run:

defmodule TtlStore do
  use GenServer

  def start_link(name) do
    GenServer.start_link(__MODULE__, name, [])
  end

  def start_link(name, opts) do
    default_ttl = Keyword.get(opts, :default_ttl, 60)
    GenServer.start_link(__MODULE__, name, [:default_ttl => default_ttl])
  end

  def put(pid, key, value) do
    validate_binary(key)
    validate_binary(value)
    GenServer.call(pid, {:put, key, value, nil})
  end

  def put(pid, key, value, opts) do
    validate_binary(key)
    validate_binary(value)
    GenServer.call(pid, {:put, key, value, opts})
  end

  def get(pid, key) do
    GenServer.call(pid, {:get, key})
  end

  def delete(pid, key) do
    GenServer.call(pid, {:delete, key})
  end

  def stats(pid) do
    if is_pid(pid) do
      GenServer.call(pid, :stats)
    else
      raise ArgumentError, "invalid pid"
    end
  end

  def handle_info(:evict, state) do
    now = System.system_time(:second)
    new_store = Enum.filter(state.store, fn {_k, {_v, expires_at}} -> expires_at > now end) |> Enum.into(%{})
    new_state = %{state | store: new_store}
    Process.send_after(self(), :evict, state.eviction_interval)
    {:noreply, new_state}
  end
end

Route decision: keep it as a long-context general driver. Do not route Elixir work through it.