{"slug": "scoring-a2a-agent-skills-with-system-one-jev-in-net", "title": "Scoring A2A Agent Skills with System One (Jev) in .NET", "summary": "A developer has built a pre-LLM gating layer in .NET that scores individual A2A agent skills against a user request using TypeSafe's System One (Jev) classification API, exposing only the winning skills to the downstream LLM tool loop. The approach flattens each agent card into scorable rubrics and sends all skill questions in a single HTTP call, returning typed relevance scores compared against a default 0.6 threshold. The developer argues this avoids re-sending full agent-card descriptions on every tool-loop iteration, which grows the prompt linearly with the number of agents.", "body_md": "Follow-up to\n\n`AgentCard` into an `AIFunction` and let one LLM loop do the routing.\n`tools = agents.Select(ToTool)` means the prompt grows linearly with the number of agents, and each tool description is a blob of the whole card:\n\n```\nAsk the SupplyChainAnalyst specialist. \nManages warehouse logistics, stock levels, inbound shipment delivery statuses,\nand DWH stock-velocity tracking.\nSkills: GetStock, GetShipments.\n```\n\nWorse, that blob is re-sent on **every** iteration of the tool loop, not once per request.\n\nTwo consequences:\n\n[System One](https://docs.typesafe.ai/api) is a classification primitive, not a chat model. You send a\n\n`state` plus a map of typed `questions`, and get one typed answer per key. Three primitives:\n\n`noul` - TypeSafe's name for the boolean primitive; returns the probability of \"yes\" rather than a yes/no token,`choice` (pick one + distribution),`score` (ordered rubric, probability-weighted).\nAnswers come back structured, so there is nothing to parse out of prose.\n\nWe use it as a **pre-LLM gate**: score each agent skill against the request, expose only the winners.\n\nThe unit of selection is the **skill**, not the agent. `ToolCatalog` flattens every card into scorable rubrics:\n\n```\nprivate static string BuildRubric(RemoteAgent agent, string name, string? description, IReadOnlyList<string>? tags)\n{\n    var rubric = string.IsNullOrWhiteSpace(description) ? name : $\"{name} — {description}\";\n    rubric = $\"{agent.Card?.Name}: {rubric}\";\n    if (tags is { Count: > 0 })\n        rubric += $\" (topics: {string.Join(\", \", tags)})\";\n    return rubric; \n}\n```\n\nOne `ScoreQuestion` per skill, all in **one** HTTP call:\n\n```\nquestions[skill.Key] = new ScoreQuestion\n{\n    Instructions = new\n    {\n        question = \"How relevant is this skill to answering the user's latest request?\",\n        skill = skill.Rubric,\n    },\n    Criteria =\n    [\n        \"Not needed; the request can be answered fully without this skill.\",\n        \"Needed; the request (or part of it) requires this skill.\",\n    ],\n};\n```\n\nUsing exactly **two** criteria makes the score a 0..1 relevance probability, directly comparable to `RelevanceThreshold` (default `0.6`). Add a third criterion and the score rescales - `0.6` silently stops meaning what it did.\n\n**Request** - `POST /v1/systemone` for *\"How much stock is left for the winter coat?\"*. The `state` also carries the last few turns, so follow-ups like *\"and the shipments?\"* still score correctly:\n\n```\n{\n  \"model\": \"jev-latest\",\n  \"state\": {\n    \"latest_user_message\": \"How much stock is left for the winter coat?\",\n    \"conversation\": [\n      { \"role\": \"user\", \"content\": \"...\" },\n      { \"role\": \"assistant\", \"content\": \"...\" }\n    ]\n  },\n  \"questions\": {\n    \"GetProduct\": {\n      \"type\": \"score\",\n      \"instructions\": {\n        \"question\": \"How relevant is this skill to answering the user's latest request?\",\n        \"skill\": \"AssortmentSpecialist: GetProduct — Look up a product's SKU, category, active status, and store coverage by name. (topics: catalog, assortment, product)\"\n      },\n      \"criteria\": [\n        \"Not needed; the request can be answered fully without this skill.\",\n        \"Needed; the request (or part of it) requires this skill.\"\n      ]\n    },\n    \"GetActiveCatalog\": { \"type\": \"score\", \"instructions\": { \"...\": \"...\" }, \"criteria\": [\"...\", \"...\"] },\n    \"GetStock\":         { \"type\": \"score\", \"instructions\": { \"...\": \"...\" }, \"criteria\": [\"...\", \"...\"] },\n    \"GetShipments\":     { \"type\": \"score\", \"instructions\": { \"...\": \"...\" }, \"criteria\": [\"...\", \"...\"] }\n  }\n}\n```\n\n**Response** - same keys, typed answers. The `jev-latest` alias resolves to a pinned version, so you can log exactly what scored:\n\n```\n{\n  \"model\": \"jev-1.13.0\",\n  \"answers\": {\n    \"GetProduct\":       { \"type\": \"score\", \"score\": 0.21, \"confidence\": 0.88, \"probabilities\": { \"0\": 0.79, \"1\": 0.21 }, \"legend\": { \"0\": \"Not needed...\", \"1\": \"Needed...\" } },\n    \"GetActiveCatalog\": { \"type\": \"score\", \"score\": 0.06, \"confidence\": 0.95, \"probabilities\": { \"0\": 0.94, \"1\": 0.06 } },\n    \"GetStock\":         { \"type\": \"score\", \"score\": 0.96, \"confidence\": 0.93, \"probabilities\": { \"0\": 0.04, \"1\": 0.96 } },\n    \"GetShipments\":     { \"type\": \"score\", \"score\": 0.44, \"confidence\": 0.61, \"probabilities\": { \"0\": 0.56, \"1\": 0.44 } }\n  },\n  \"usage\": { \"input_tokens\": 512, \"output_tokens\": 24 }\n}\n```\n\n`GetStock` clears `0.6`, so **only `SupplyChainAnalyst`** becomes a tool. The assortment agent is never offered. Ask *\"which stores carry it, and how much stock is left?\"* and `GetProduct` also clears - both agents are exposed, and the parallel fan-out from part 1 still happens.\n\nFewer tools, and a narrower description per surviving tool - built only from the skills that scored:\n\n``` js\nif (selectedSkills.TryGetValue(agent.Card.Name!, out var kept) && kept.Count > 0)\n    return $\"Ask the {agent.Card.Name} specialist. Relevant capabilities: {skillText}\";\n\n// TypeSafe off / fallback → full card + all skills (part-1 behavior)\nreturn $\"Ask the {agent.Card.Name} specialist. {agent.Card.Description} Skills: {allSkills}.\";\n```\n\nOne step inserted before the loop. `OrchestrationService` gains an `IToolSelector`:\n\n``` js\nvar catalog   = ToolCatalog.FromAgents(await registry.GetAgents(ct));\nvar selection = await toolSelector.SelectAsync(catalog, userMessage, thread.Turns, ct);\nvar tools     = selection.Agents.Select(a => ToTool(a, selection.SelectedSkills)).Cast<AITool>().ToList();\nphp\ngraph LR\n    User([User]) --> Cat[ToolCatalog<br/>cards → skill rubrics]\n    Cat --> Sel{{IToolSelector}}\n    Sel -. score skills .-> TS[(System One)]\n    Sel -->|surviving tools only| LLM[LLM tool loop]\n    LLM -- A2A --> A[Assortment]\n    LLM -- A2A --> S[SupplyChain]\n```\n\n`IToolSelector` is an interface for a reason: without an API key the app registers a pass-through\n\n`AllToolsSelector` and behaves exactly like part 1, so the gate is also its own off-switch.", "url": "https://wpnews.pro/news/scoring-a2a-agent-skills-with-system-one-jev-in-net", "canonical_source": "https://dev.to/ohalay/scoring-a2a-agent-skills-with-system-one-jev-in-net-k1n", "published_at": "2026-09-21 16:06:50+00:00", "updated_at": "2026-09-21 16:32:53.764765+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools", "large-language-models"], "entities": ["TypeSafe", "System One", "Jev", ".NET", "A2A", "AgentCard", "AIFunction", "ToolCatalog"], "alternates": {"html": "https://wpnews.pro/news/scoring-a2a-agent-skills-with-system-one-jev-in-net", "markdown": "https://wpnews.pro/news/scoring-a2a-agent-skills-with-system-one-jev-in-net.md", "text": "https://wpnews.pro/news/scoring-a2a-agent-skills-with-system-one-jev-in-net.txt", "jsonld": "https://wpnews.pro/news/scoring-a2a-agent-skills-with-system-one-jev-in-net.jsonld"}}