{"slug": "a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass", "title": "A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification", "summary": "Researchers at ACL 2026 introduced a method to reuse LLM hidden states for classification tasks, training lightweight probes on token- and layer-selective representations to eliminate separate guard models. Their two-stage aggregator, tested on Llama-3.2-3B, GPT-OSS-20B, and Qwen3-30B-A3B, achieved competitive accuracy with lower latency and VRAM usage compared to traditional pipelines.", "body_md": "##### Abstract\n\nProduction LLM systems often rely on separate models for safety and other classification-heavy steps, increasing latency, VRAM footprint, and operational complexity. We instead reuse computation already paid for by the serving LLM: we train lightweight probes on its hidden states and predict labels in the same forward pass used for generation. We frame classification as representation selection over the full token×layer hidden-state tensor, rather than committing to a fixed token or fixed layer (e.g., first-token logits or final-layer pooling). To implement this, we introduce a two-stage aggregator that (i) summarizes tokens within each layer and (ii) aggregates across layer summaries to form a single representation for classification. We instantiate this template with direct pooling, a 100K-parameter scoring-attention gate, and a downcast multi-head self-attention (MHA) probe with up to 35M trainable parameters. Across safety and sentiment benchmarks our probes improve over logit-only reuse (e.g., MULI) and are competitive with substantially larger task-specific baselines, while preserving near-serving latency and avoiding the VRAM and latency costs of a separate guard-model pipeline. Multi-backbone experiments on dense and mixture-of-experts architectures (Llama-3.2-3B, GPT-OSS-20B, Qwen3-30B-A3B) confirm that these findings generalize beyond a single model family.- Anthology ID:\n- 2026.acl-long.1955\n- Volume:\n[Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](/volumes/2026.acl-long/)- Month:\n- July\n- Year:\n- 2026\n- Address:\n- San Diego, California, United States\n- Editors:\n[Maria Liakata](/people/maria-liakata/),[Viviane P. Moreira](/people/viviane-p-moreira/unverified/),[Jiajun Zhang](/people/jiajun-zhang/unverified/),[David Jurgens](/people/david-jurgens/)- Venue:\n[ACL](/venues/acl/)- SIG:\n- Publisher:\n- Association for Computational Linguistics\n- Note:\n- Pages:\n- 42226–42239\n- Language:\n- URL:\n[https://aclanthology.org/2026.acl-long.1955/](https://aclanthology.org/2026.acl-long.1955/)- DOI:\n- Cite (ACL):\n- Gonzalo Ariel Meyoyan and Luciano Del Corro. 2026.\n[A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification](https://aclanthology.org/2026.acl-long.1955/). In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 42226–42239, San Diego, California, United States. Association for Computational Linguistics. - Cite (Informal):\n[A BERTology View of LLM Orchestrations: Token- and Layer-Selective Probes for Efficient Single-Pass Classification](https://aclanthology.org/2026.acl-long.1955/)(Meyoyan & Del Corro, ACL 2026)- PDF:\n[https://aclanthology.org/2026.acl-long.1955.pdf](https://aclanthology.org/2026.acl-long.1955.pdf)", "url": "https://wpnews.pro/news/a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass", "canonical_source": "https://aclanthology.org/2026.acl-long.1955/", "published_at": "2026-06-22 00:00:00+00:00", "updated_at": "2026-06-26 08:16:54.675050+00:00", "lang": "en", "topics": ["large-language-models", "natural-language-processing", "ai-safety", "ai-research"], "entities": ["ACL", "Llama-3.2-3B", "GPT-OSS-20B", "Qwen3-30B-A3B", "Gonzalo Ariel Meyoyan", "Luciano Del Corro", "Association for Computational Linguistics"], "alternates": {"html": "https://wpnews.pro/news/a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass", "markdown": "https://wpnews.pro/news/a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass.md", "text": "https://wpnews.pro/news/a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass.txt", "jsonld": "https://wpnews.pro/news/a-bertology-view-of-llm-orchestrations-token-and-layer-selective-probes-for-pass.jsonld"}}