{"slug": "how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb", "title": "How to run LFM2.5-24B-A2B Q8_0 on 2× RTX 5060 Ti 16 GB", "summary": "A benchmark of LFM2.5-24B-A2B Q8_0 running on 2× RTX 5060 Ti 16 GB GPUs found a 0/3 delivered-pipeline pass rate across three v5 runs on the 32 GB tensor-split tier, each capped at 130,000 tokens, despite all three runs finishing cleanly with review, module and tests. The same setup passed 30 of 30 needle-in-a-haystack probes, including at 130,864 prompt tokens, with prefill decaying from 8,070 to 5,453 tok/s and decode from 149 to 104 tok/s as the context window filled. The author reports a 76-79% thinking tax and 64-71% eight-gram duplication in emitted characters and concludes the model should be kept as a long-context general driver, not routed Elixir work through it.", "body_md": "**Warning:** this is a temporary briefing, written by an LLM and not yet evaluated or rewritten by a human to get rid of the signature AI-slop writing style. It’s published as a draft despite its shortcomings, so that you can find out how to run it and what to expect in terms of `pp` and `tg` on this hardware, as well as what to expect in terms of coding performance on this benchmark, *grosso modo*.\n\nThree v5 runs on the 32 GB tensor-split tier, each capped at 130,000 tokens — near the full 131,072 served slot. The cap tracks the window on purpose: a short mid-answer cap can retire a model that would have finished, so a run is only comparable to the slot it was given.\n\nAll three runs finished cleanly with review, module and tests — and none produced a runnable pipeline. Two modules do not compile; the third compiles while its test file references handles that were never bound. Delivered-pipeline pass rate: 0 / 3. Thinking tax ran 76-79 % of emitted characters, with 64-71 % eight-gram duplication across the whole answer — repetitive reasoning, never a terminal loop.\n\nThe needle ladder is the other half of the story. One unique anchor, buried at every tenth of the served slot, three repeats per depth, each request opening with a fresh nonce so every step measures cold prefill. Thirty of thirty hit, including at 130,864 prompt tokens. Prefill decays smoothly from 8,070 to 5,453 tok/s and decode from 149 to 104 tok/s as the window fills — a soft curve, not a cliff.\n\nMethodology, disclosed: the first needle attempt used a 48-token answer cap and returned empty content at every depth — archived, not measured. The 90 % and 100 % steps initially failed with a context-overflow error because a byte-based token estimate undershot the real count; the runner now feeds the server’s reported token count back into the estimate, and the re-run converged within 0.02 % of the target depth.\n\n## Representative output\n\nRun 1’s `TtlStore` — trimmed to the public API and eviction handler. The full module plus its ExUnit suite shipped fenced; this is the parts that actually run:\n\n``` python\ndefmodule TtlStore do\n  use GenServer\n\n  def start_link(name) do\n    GenServer.start_link(__MODULE__, name, [])\n  end\n\n  def start_link(name, opts) do\n    default_ttl = Keyword.get(opts, :default_ttl, 60)\n    GenServer.start_link(__MODULE__, name, [:default_ttl => default_ttl])\n  end\n\n  def put(pid, key, value) do\n    validate_binary(key)\n    validate_binary(value)\n    GenServer.call(pid, {:put, key, value, nil})\n  end\n\n  def put(pid, key, value, opts) do\n    validate_binary(key)\n    validate_binary(value)\n    GenServer.call(pid, {:put, key, value, opts})\n  end\n\n  def get(pid, key) do\n    GenServer.call(pid, {:get, key})\n  end\n\n  def delete(pid, key) do\n    GenServer.call(pid, {:delete, key})\n  end\n\n  def stats(pid) do\n    if is_pid(pid) do\n      GenServer.call(pid, :stats)\n    else\n      raise ArgumentError, \"invalid pid\"\n    end\n  end\n\n  def handle_info(:evict, state) do\n    now = System.system_time(:second)\n    new_store = Enum.filter(state.store, fn {_k, {_v, expires_at}} -> expires_at > now end) |> Enum.into(%{})\n    new_state = %{state | store: new_store}\n    Process.send_after(self(), :evict, state.eviction_interval)\n    {:noreply, new_state}\n  end\nend\n```\n\nRoute decision: keep it as a long-context general driver. Do not route Elixir work through it.", "url": "https://wpnews.pro/news/how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb", "canonical_source": "https://overbring.com/ai-benchmarks/2026-10-31-unsloth-lfm2.5-24b-a2b-q8_0-gguf-rtx-5060-ti-16gb-x2-tensor-split-llama-server-cuda/", "published_at": "2026-10-31 00:43:07+00:00", "updated_at": "2026-11-09 01:58:40.689238+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-research"], "entities": ["LFM2.5-24B-A2B", "RTX 5060 Ti", "Elixir", "ExUnit", "TtlStore"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb", "markdown": "https://wpnews.pro/news/how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb.md", "text": "https://wpnews.pro/news/how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb.txt", "jsonld": "https://wpnews.pro/news/how-to-run-lfm2-5-24b-a2b-q8-0-on-2x-rtx-5060-ti-16-gb.jsonld"}}