{"slug": "request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow", "title": "Request to add sriram1983007/sra-stablecoin-risk-bench to the benchmark allow-list", "summary": "A Hugging Face user submitted the sriram1983007/sra-stablecoin-risk-bench dataset for inclusion in the platform's benchmark allow-list, asking the Hugging Face team to add it per the Evaluation Results documentation. The benchmark measures how models handle stablecoin (USDC) payments made by AI agents across two held-out test tasks: risk_gate (1,175 cases, including 69 prompt injections, where a model approves, holds or rejects a payment) and dispute (825 cases, where a model picks the remedy possible given where a payment sits relative to settlement finality). The dataset uses the inspect-ai evaluation framework with eval_risk_gate and eval_dispute configs, temperature 0 generation, a pattern scorer on the decision or outcome field, a full scorer in score.py covering macro-F1, unsafe approvals, wrongful refunds and denials, and is licensed CC-BY-4.0.", "body_md": "Hi Hugging Face team,\n\nThe Evaluation Results docs say to get in touch to add a dataset to the benchmark allow-list, so I’m asking here.\n\nDataset (eval.yaml is in the repo root): [sriram1983007/sra-stablecoin-risk-bench · Datasets at Hugging Face](https://huggingface.co/datasets/sriram1983007/sra-stablecoin-risk-bench)\n\nWhat it measures: how a model handles stablecoin (USDC) payments made by AI agents. Two tasks, both with held-out test splits:\n\n- risk_gate (1,175 cases): approve, hold or reject a payment, given the policy, the payment request (treated as untrusted, including 69 prompt injections) and the results of verification tools.\n- dispute (825 cases): choose the remedy that is actually possible given where the payment sits relative to settlement finality.\n\nSetup:\n\n- eval.yaml uses evaluation_framework: inspect-ai, with configs eval_risk_gate and eval_dispute, split test, a system-message solver with the task prompt, generate at temperature 0, and a pattern scorer on the decision or outcome field.\n- The full scorer (macro-F1, unsafe approvals, wrongful refunds and denials) is score.py in the same repo.\n- License CC-BY-4.0.", "url": "https://wpnews.pro/news/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow", "canonical_source": "https://discuss.huggingface.co/t/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow-list/190963#post_1", "published_at": "2026-10-24 09:02:35+00:00", "updated_at": "2026-10-24 09:12:35.235366+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "ai-research"], "entities": ["Hugging Face", "sriram1983007/sra-stablecoin-risk-bench", "inspect-ai", "USDC"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow", "markdown": "https://wpnews.pro/news/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow.md", "text": "https://wpnews.pro/news/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow.txt", "jsonld": "https://wpnews.pro/news/request-to-add-sriram1983007-sra-stablecoin-risk-bench-to-the-benchmark-allow.jsonld"}}