cd /news/ai-safety/request-to-add-sriram1983007-sra-sta… · home › topics › ai-safety › article
[ARTICLE · art-156671] src=discuss.huggingface.co ↗ pub= topic=ai-safety verified=true sentiment=· neutral

Request to add sriram1983007/sra-stablecoin-risk-bench to the benchmark allow-list

A Hugging Face user submitted the sriram1983007/sra-stablecoin-risk-bench dataset for inclusion in the platform's benchmark allow-list, asking the Hugging Face team to add it per the Evaluation Results documentation. The benchmark measures how models handle stablecoin (USDC) payments made by AI agents across two held-out test tasks: risk_gate (1,175 cases, including 69 prompt injections, where a model approves, holds or rejects a payment) and dispute (825 cases, where a model picks the remedy possible given where a payment sits relative to settlement finality). The dataset uses the inspect-ai evaluation framework with eval_risk_gate and eval_dispute configs, temperature 0 generation, a pattern scorer on the decision or outcome field, a full scorer in score.py covering macro-F1, unsafe approvals, wrongful refunds and denials, and is licensed CC-BY-4.0.

read1 min views1 publishedOct 24, 2026

Hi Hugging Face team,

The Evaluation Results docs say to get in touch to add a dataset to the benchmark allow-list, so I’m asking here.

Dataset (eval.yaml is in the repo root): sriram1983007/sra-stablecoin-risk-bench · Datasets at Hugging Face What it measures: how a model handles stablecoin (USDC) payments made by AI agents. Two tasks, both with held-out test splits:

  • risk_gate (1,175 cases): approve, hold or reject a payment, given the policy, the payment request (treated as untrusted, including 69 prompt injections) and the results of verification tools.
  • dispute (825 cases): choose the remedy that is actually possible given where the payment sits relative to settlement finality.

Setup:

  • eval.yaml uses evaluation_framework: inspect-ai, with configs eval_risk_gate and eval_dispute, split test, a system-message solver with the task prompt, generate at temperature 0, and a pattern scorer on the decision or outcome field.
  • The full scorer (macro-F1, unsafe approvals, wrongful refunds and denials) is score.py in the same repo.
  • License CC-BY-4.0.
── more in #ai-safety 4 stories · sorted by recency
── more on @hugging face 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/request-to-add-srira…] indexed:0 read:1min 2026-10-24 · —