{"slug": "marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems", "title": "MARQ: A Modular, LLM-Agnostic Requirement Quality Assessor Infused with Systems Engineering Knowledge", "summary": "A modular, LLM-agnostic requirement quality assessor called MARQ, which supplements large language models with a criterion-specific knowledge stack operationalizing INCOSE requirement quality criteria, raised median F1 scores on an expert-labeled NASA requirement set across three models over 100 executions each, according to the arXiv paper 2610.36234v1. MARQ scored median F1 of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol, and 0.53 for Gemma 3 12B, versus off-the-shelf baselines of 0.55, 0.52, and 0.18 respectively. The authors report the gains are model-dependent — Claude Opus 5 improved defect detection and false-alarm control, GPT-5.6 Sol gained detection and execution-level consistency at the cost of more false alarms, and Gemma 3 12B matched off-the-shelf proprietary models while remaining highly variable — with improvements concentrated in Correctness and Conformance criteria.", "body_md": "arXiv:2610.36234v1 Announce Type: new \nAbstract: Requirements engineering (RE) is foundational to systems engineering (SE), as requirement defects can propagate downstream and adversely affect development outcomes. Although large language models (LLMs) are promising for requirement quality assessment, prior work shows that off-the-shelf models miss a large share of expert-identified defects, perform unevenly across quality criteria, and vary across executions. This study investigates whether infusing expert-derived SE knowledge can improve LLM-based requirement quality evaluation beyond off-the-shelf capability. We present MARQ, a modular, LLM-agnostic prototype that supplements LLMs with a criterion-specific knowledge stack operationalizing INCOSE requirement quality criteria and evaluated against off-the-shelf baselines on an expert-labeled NASA requirement set using Claude Opus 5, GPT-5.6 Sol, and the open-weight Gemma 3 12B, with 100 executions per model. MARQ achieves median F1 scores of 0.80 for Claude Opus 5, 0.62 for GPT-5.6 Sol, and 0.53 for Gemma 3 12B, compared with off-the-shelf baselines of 0.55, 0.52, and 0.18, respectively. However, the nature of these gains is model-dependent. Claude Opus 5 improves defect detection and false-alarm control; GPT-5.6 Sol gains detection and better execution-level consistency at the cost of more false alarms; and Gemma 3 12B reaches the performance of off-the-shelf proprietary models, although it remains highly variable. Criterion-level analysis shows gains concentrated in Correctness and Conformance, trade-offs between misses and false alarms within individual criteria, and model-dependent responses across several criteria. These findings position SE knowledge infusion as a promising model-agnostic pathway for strengthening trustworthy LLM-based RE support while underscoring the need for criterion-specific interventions and human-in-the-loop review.", "url": "https://wpnews.pro/news/marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems", "canonical_source": "https://arxiv.org/abs/2610.36234", "published_at": "2026-10-28 04:00:00+00:00", "updated_at": "2026-10-28 04:11:40.685431+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-tools"], "entities": ["MARQ", "INCOSE", "NASA", "Claude Opus 5", "GPT-5.6 Sol", "Gemma 3 12B", "arXiv"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems", "markdown": "https://wpnews.pro/news/marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems.md", "text": "https://wpnews.pro/news/marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems.txt", "jsonld": "https://wpnews.pro/news/marq-a-modular-llm-agnostic-requirement-quality-assessor-infused-with-systems.jsonld"}}