{"slug": "pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026", "title": "PCB-Bench: Benchmarking LLMs for PCB Placement and Routing (ICLR 2026)", "summary": "Researchers introduced PCB-Bench, the first comprehensive benchmark for evaluating large language models on printed circuit board placement and routing tasks. The benchmark includes over 3,700 questions across three task settings, covering text-only, multimodal, and design understanding scenarios, and tests models on real-world PCB engineering knowledge. Initial evaluations of state-of-the-art LLMs revealed significant gaps in domain-specific reasoning, highlighting the need for specialized AI tools in hardware design.", "body_md": "**PCB-Bench: Benchmarking LLMs for Printed Circuit Board Placement and Routing** (ICLR 2026)\n\n[📄 [OpenReview]: https://openreview.net/forum?id=Q5QLu7XTWx&referrer](https://openreview.net/forum?id=Q5QLu7XTWx&referrer)\n\n[🌐 [Project Page]: https://digailab.github.io/PCB-Bench/](https://digailab.github.io/PCB-Bench/)\n\nPCB-Bench is the **first comprehensive benchmark** designed to systematically evaluate (multimodal) large language models (LLMs/MLLMs) in the context of **PCB placement and routing**. It addresses the lack of standardized benchmarks and high-fidelity datasets for real-world PCB engineering reasoning by integrating **text**, **images**, and **real PCB design artifacts** into a unified evaluation framework.\n\nPCB-Bench spans **three complementary task settings** and corresponding datasets:\n\n- ~\n**1,800** expert-written**free-form QA** instances - Each QA has a corresponding\n**single-choice question (CQ)** version - Total ~\n**3,700** questions (QA + CQ) - Covers\n**component placement**,** routing strategies**, and** design rule compliance** - Covers both\n**macro-level**(global design principles) and** micro-level**(fine-grained implementation details), across placement and routing, with topic labels (e.g., signal integrity, EMI/EMC, power planning, differential pairs, DFM, etc.).\n\n- ~\n**500** problems requiring joint interpretation of**PCB layout images + technical prompts** - Includes\n**choice questions**,** cloze-style fill-in-the-blank**, and** free-form QA** - Covers visual-semantic subtasks such as component identification, functional block recognition, trace reasoning, via presence checking, differential-pair continuity analysis, etc.\n\n**174** complete real-world PCB projects collected from**OSHWHub (operated by JLCPCB)**([https://oshwhub.com/](https://oshwhub.com/))- Each design includes artifacts such as\n**schematics**,** placement/routing files**,** design descriptions**,** component libraries**, and** EDA software screenshots** - Task setting: given a\n**standalone EDA editor screenshot**(no extra text/schematic provided), models generate a** free-form description**of the board’s function/structure/application scenario, assessing structured visual interpretation of professional PCB artifacts.\n\nPCB-Bench is organized into three task settings aligned with real engineering workflows:\n\n-\n**Task 1: Text-to-Text QA & CQ**\n\nEvaluate PCB placement/routing knowledge via both open-ended generation and objective multiple-choice selection. -\n**Task 2: Image-and-Text Multimodal QA & CQ**\n\nAnswer questions based on PCB layout images together with textual prompts. -\n**Task 3: PCB Design Understanding (Screenshot-to-Description)**\n\nDescribe full-board PCB screenshots from EDA tools using free-form functional/structural descriptions.\n\nAll models are evaluated under a **unified zero-shot setting** across tasks (each instance is answered independently, without demonstrations or fine-tuning).\n\n**Choice Questions (CQ):** Top-1**Accuracy****Free-form QA:****BERTScore** and**Sentence-BERT (SBERT) similarity** for semantic consistency with reference answers**Task 3 (Design Understanding):** additionally report**Precision / Recall / F1-score** to capture complementary aspects of prediction quality\n\nThe paper benchmarks a diverse set of state-of-the-art LLMs/MLLMs under the unified protocol, including frontier and open-source models; and additionally evaluates domain-specific variants derived from Qwen2.5-7B-Instruct to study PCB-oriented specialization.\n\n(For the exact model lists per task, please refer to the paper.)\n\n- PCB designs are collected from\n**publicly available and legally accessible sources**, with** no proprietary or sensitive industrial data**involved. - Real-world PCB projects are collected from\n**OSHWHub**([https://oshwhub.com/](https://oshwhub.com/)); each design is associated with a corresponding URL link to ensure transparency and IP protection. - PCB-Bench is released with\n**open licensing** to support reproducibility and standardized comparison.\n\nThe paper details task formulations, metrics, and model settings. Results are obtained under the unified **zero-shot** setting. The benchmark is released **along with evaluation scripts and configuration files** to support reproduction and extension.\n\nWe gratefully acknowledge the support of Qiyunfang Technology Co., LTD. We also thank OSHWHub and JLCPCB for providing access to publicly available PCB design resources.\n\nIf you use PCB-Bench in your research, please cite:\n\n```\n@inproceedings{lipcb,\n  title     = {PCB-Bench: Benchmarking LLMs for Printed Circuit Board Placement and Routing},\n  author    = {Li, Jindong and Chen, Lianrong and Yang, Bin and Zhu, Jiadong and Wang, Ying and Ma, Yuzhe and Yang, Menglin},\n  booktitle = {The Fourteenth International Conference on Learning Representations},\n  year      = {2026}\n}\n```\n\n", "url": "https://wpnews.pro/news/pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026", "canonical_source": "https://github.com/digailab/PCB-Bench", "published_at": "2026-06-29 03:43:21+00:00", "updated_at": "2026-06-29 03:57:57.566664+00:00", "lang": "en", "topics": ["large-language-models", "artificial-intelligence", "ai-research", "ai-tools", "computer-vision"], "entities": ["PCB-Bench", "OSHWHub", "JLCPCB", "Qwen2.5-7B-Instruct", "ICLR 2026", "OpenReview"], "alternates": {"html": "https://wpnews.pro/news/pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026", "markdown": "https://wpnews.pro/news/pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026.md", "text": "https://wpnews.pro/news/pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026.txt", "jsonld": "https://wpnews.pro/news/pcb-bench-benchmarking-llms-for-pcb-placement-and-routing-iclr-2026.jsonld"}}