Hi Hugging Face team,
The Evaluation Results docs say to get in touch to add a dataset to the benchmark allow-list, so I’m asking here.
Dataset (eval.yaml is in the repo root): sriram1983007/sra-stablecoin-risk-bench · Datasets at Hugging Face What it measures: how a model handles stablecoin (USDC) payments made by AI agents. Two tasks, both with held-out test splits:
- risk_gate (1,175 cases): approve, hold or reject a payment, given the policy, the payment request (treated as untrusted, including 69 prompt injections) and the results of verification tools.
- dispute (825 cases): choose the remedy that is actually possible given where the payment sits relative to settlement finality.
Setup:
- eval.yaml uses evaluation_framework: inspect-ai, with configs eval_risk_gate and eval_dispute, split test, a system-message solver with the task prompt, generate at temperature 0, and a pattern scorer on the decision or outcome field.
- The full scorer (macro-F1, unsafe approvals, wrongful refunds and denials) is score.py in the same repo.
- License CC-BY-4.0.