Scoring A2A Agent Skills with System One (Jev) in .NET

A developer has built a pre-LLM gating layer in .NET that scores individual A2A agent skills against a user request using TypeSafe's System One (Jev) classification API, exposing only the winning skills to the downstream LLM tool loop. The approach flattens each agent card into scorable rubrics and sends all skill questions in a single HTTP call, returning typed relevance scores compared against a default 0.6 threshold. The developer argues this avoids re-sending full agent-card descriptions on every tool-loop iteration, which grows the prompt linearly with the number of agents.

Follow-up to AgentCard into an AIFunction and let one LLM loop do the routing. tools = agents.Select ToTool means the prompt grows linearly with the number of agents, and each tool description is a blob of the whole card: Ask the SupplyChainAnalyst specialist. Manages warehouse logistics, stock levels, inbound shipment delivery statuses, and DWH stock-velocity tracking. Skills: GetStock, GetShipments. Worse, that blob is re-sent on every iteration of the tool loop, not once per request. Two consequences: System One https://docs.typesafe.ai/api is a classification primitive, not a chat model. You send a state plus a map of typed questions , and get one typed answer per key. Three primitives: noul - TypeSafe's name for the boolean primitive; returns the probability of "yes" rather than a yes/no token, choice pick one + distribution , score ordered rubric, probability-weighted . Answers come back structured, so there is nothing to parse out of prose. We use it as a pre-LLM gate : score each agent skill against the request, expose only the winners. The unit of selection is the skill , not the agent. ToolCatalog flattens every card into scorable rubrics: private static string BuildRubric RemoteAgent agent, string name, string? description, IReadOnlyList<string ? tags { var rubric = string.IsNullOrWhiteSpace description ? name : $"{name} — {description}"; rubric = $"{agent.Card?.Name}: {rubric}"; if tags is { Count: 0 } rubric += $" topics: {string.Join ", ", tags } "; return rubric; } One ScoreQuestion per skill, all in one HTTP call: questions skill.Key = new ScoreQuestion { Instructions = new { question = "How relevant is this skill to answering the user's latest request?", skill = skill.Rubric, }, Criteria = "Not needed; the request can be answered fully without this skill.", "Needed; the request or part of it requires this skill.", , }; Using exactly two criteria makes the score a 0..1 relevance probability, directly comparable to RelevanceThreshold default 0.6 . Add a third criterion and the score rescales - 0.6 silently stops meaning what it did. Request - POST /v1/systemone for "How much stock is left for the winter coat?" . The state also carries the last few turns, so follow-ups like "and the shipments?" still score correctly: { "model": "jev-latest", "state": { "latest user message": "How much stock is left for the winter coat?", "conversation": { "role": "user", "content": "..." }, { "role": "assistant", "content": "..." } }, "questions": { "GetProduct": { "type": "score", "instructions": { "question": "How relevant is this skill to answering the user's latest request?", "skill": "AssortmentSpecialist: GetProduct — Look up a product's SKU, category, active status, and store coverage by name. topics: catalog, assortment, product " }, "criteria": "Not needed; the request can be answered fully without this skill.", "Needed; the request or part of it requires this skill." }, "GetActiveCatalog": { "type": "score", "instructions": { "...": "..." }, "criteria": "...", "..." }, "GetStock": { "type": "score", "instructions": { "...": "..." }, "criteria": "...", "..." }, "GetShipments": { "type": "score", "instructions": { "...": "..." }, "criteria": "...", "..." } } } Response - same keys, typed answers. The jev-latest alias resolves to a pinned version, so you can log exactly what scored: { "model": "jev-1.13.0", "answers": { "GetProduct": { "type": "score", "score": 0.21, "confidence": 0.88, "probabilities": { "0": 0.79, "1": 0.21 }, "legend": { "0": "Not needed...", "1": "Needed..." } }, "GetActiveCatalog": { "type": "score", "score": 0.06, "confidence": 0.95, "probabilities": { "0": 0.94, "1": 0.06 } }, "GetStock": { "type": "score", "score": 0.96, "confidence": 0.93, "probabilities": { "0": 0.04, "1": 0.96 } }, "GetShipments": { "type": "score", "score": 0.44, "confidence": 0.61, "probabilities": { "0": 0.56, "1": 0.44 } } }, "usage": { "input tokens": 512, "output tokens": 24 } } GetStock clears 0.6 , so only SupplyChainAnalyst becomes a tool. The assortment agent is never offered. Ask "which stores carry it, and how much stock is left?" and GetProduct also clears - both agents are exposed, and the parallel fan-out from part 1 still happens. Fewer tools, and a narrower description per surviving tool - built only from the skills that scored: js if selectedSkills.TryGetValue agent.Card.Name , out var kept && kept.Count 0 return $"Ask the {agent.Card.Name} specialist. Relevant capabilities: {skillText}"; // TypeSafe off / fallback → full card + all skills part-1 behavior return $"Ask the {agent.Card.Name} specialist. {agent.Card.Description} Skills: {allSkills}."; One step inserted before the loop. OrchestrationService gains an IToolSelector : js var catalog = ToolCatalog.FromAgents await registry.GetAgents ct ; var selection = await toolSelector.SelectAsync catalog, userMessage, thread.Turns, ct ; var tools = selection.Agents.Select a = ToTool a, selection.SelectedSkills .Cast<AITool .ToList ; php graph LR User User -- Cat ToolCatalog<br/ cards → skill rubrics Cat -- Sel{{IToolSelector}} Sel -. score skills .- TS System One Sel -- |surviving tools only| LLM LLM tool loop LLM -- A2A -- A Assortment LLM -- A2A -- S SupplyChain IToolSelector is an interface for a reason: without an API key the app registers a pass-through AllToolsSelector and behaves exactly like part 1, so the gate is also its own off-switch.