{"slug": "nl-pddl-bench", "title": "NL-PDDL-Bench", "summary": "Researchers introduced NL-PDDL-Bench, a benchmark for converting natural language into PDDL planning specifications, and a planner-in-the-loop framework that improves executability and safety. Their method combines supervised fine-tuning, preference optimization, and inference-time repair, achieving significant gains in planner success and plan-level consistency across model families.", "body_md": "Planning often requires symbolic specifications that are both executable and verifiable. For large language models deployed in autonomous or decision-support systems, failures in such formalization may lead to unverifiable decisions, execution failures, or unsafe downstream behavior. We present NL-PDDL-Bench, a multi-domain benchmark for natural-language-to-PDDL specification construction with planner-verified executability and controlled difficulty scaling by object count. We further propose a planner-in-the-loop framework that uses validator and planner diagnostics to revise non-executable specifications through localized edits. Building on this infrastructure, we develop a planner-grounded optimization recipe that combines parameter-efficient Low-Rank Adaptation supervised fine-tuning, offline planner-derived preference pairs for Direct Preference Optimization, and inference-time planner-in-the-loop repair, without requiring online planner calls during training. We also provide a unified evaluation suite for parseability, solvability, specification similarity, and outcome-aware plan-level consistency against planner references. Experiments on representative model families show substantial gains in planner success and plan-level agreement, with improved robustness under difficulty scaling and cross-domain variation. These results highlight the value of externally verifiable formalization for reliable deployment of LLMs in safety- or security-sensitive planning systems. Code and data are available at: https://github.com/ibasicplan/NL-PDDL-Bench Category: Uncategorized. Imported rows: 9. Top imported result: Llama3.1-8B, rank 1, 81.20.", "url": "https://wpnews.pro/news/nl-pddl-bench", "canonical_source": "https://benchmarklist.com/benchmarks/nl_pddl_bench/", "published_at": "2026-06-29 00:00:00+00:00", "updated_at": "2026-07-15 19:01:05.868440+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-safety", "ai-research", "natural-language-processing"], "entities": ["NL-PDDL-Bench", "Llama3.1-8B"], "alternates": {"html": "https://wpnews.pro/news/nl-pddl-bench", "markdown": "https://wpnews.pro/news/nl-pddl-bench.md", "text": "https://wpnews.pro/news/nl-pddl-bench.txt", "jsonld": "https://wpnews.pro/news/nl-pddl-bench.jsonld"}}