{"slug": "pos-signature-demonstration-retrieval-for-in-context-dependency-parsing", "title": "POS-Signature Demonstration Retrieval for In-Context Dependency Parsing", "summary": "A POS-signature demonstration-retrieval policy produced the highest labeled attachment scores among tested prompting systems for in-context dependency parsing, reaching 72.4 and 73.2 on the 2,077-sentence UD English Web Treebank test set with two open-weight language models, according to an arXiv paper (2610.31966v1). The POS-signature approach, which represents sentences with weighted part-of-speech statistics and retrieves three length-compatible gold-parsed examples, outperformed semantic retrieval by 1.9 and 2.7 points, with paired-bootstrap 95% confidence intervals excluding zero for both contrasts. The authors released the prompts, archived outputs, and evaluation code, and note the results are exploratory and favor the approach on English-EWT.", "body_md": "arXiv:2610.31966v1 Announce Type: new \nAbstract: Demonstrations for in-context learning are often retrieved by semantic similarity, which may be a poor proxy for relevance in structured prediction. We study demonstration selection for Universal Dependencies parsing with a POS-signature policy that represents sentences with weighted part-of-speech statistics and retrieves three length-compatible, gold-parsed examples from the labeled training split. We evaluate two open-weight language models without fine-tuning on all 2,077 sentences of the UD English Web Treebank test set, comparing the policy with shot- and format-matched fixed and semantic three-shot baselines, as well as zero-shot, chain-of-thought, and critique-refinement systems. POS-signature retrieval yields the highest labeled attachment score among the tested prompting systems for both models (72.4 and 73.2), outperforming semantic retrieval by 1.9 and 2.7 points; paired-bootstrap 95% confidence intervals exclude zero for both contrasts. Chain-of-thought and refinement comparisons vary across models. These exploratory results favor the tested POS-signature retrieval approach on English-EWT. We release the prompts, archived outputs, and evaluation code.", "url": "https://wpnews.pro/news/pos-signature-demonstration-retrieval-for-in-context-dependency-parsing", "canonical_source": "https://www.machinebrief.com/news/pos-signature-demonstration-retrieval-for-in-context-depende-ekd1", "published_at": "2026-10-26 04:00:00+00:00", "updated_at": "2026-10-26 04:41:38.089250+00:00", "lang": "en", "topics": ["natural-language-processing", "large-language-models", "ai-research", "machine-learning"], "entities": ["arXiv", "Universal Dependencies", "UD English Web Treebank"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/pos-signature-demonstration-retrieval-for-in-context-dependency-parsing", "markdown": "https://wpnews.pro/news/pos-signature-demonstration-retrieval-for-in-context-dependency-parsing.md", "text": "https://wpnews.pro/news/pos-signature-demonstration-retrieval-for-in-context-dependency-parsing.txt", "jsonld": "https://wpnews.pro/news/pos-signature-demonstration-retrieval-for-in-context-dependency-parsing.jsonld"}}