# AX-RAY: VIDRAFT's Open AI Safety Diagnostic Leaderboard & Dataset Now on Hugging Face > Source: > Published: 2026-08-19 07:01:19+00:00 TL;DR:VIDRAFT has publicly released AX-RAY, a 117-item AI safety diagnostic framework, along with an open leaderboard and evaluation dataset on Hugging Face. The system detects "Causal Leakage" — a subtle failure mode where LLMs are influenced by hidden or unintended causal cues rather than legitimate reasoning paths. Two publicly available general-purpose AI models showed anomalous signals on the Causal Leakage dimension, including one from NVIDIA. AX-RAY is VIDRAFT's AI safety evaluation framework designed to diagnose latent risk behaviors and safety vulnerabilities in large language models (LLMs) and AI agents. It is structured around **117 diagnostic evaluation items** and goes beyond capability benchmarking to assess a model's potential for norm-violating or unsafe behavior. Key distinguishing features of AX-RAY: The intended scope is broad: finance, healthcare, robotics, and public services — domains where LLM misbehavior against legal or normative standards carries real-world consequences. At a conceptual level, AX-RAY probes models with structured evaluation items designed to surface latent behavioral patterns that standard capability benchmarks miss. **Causal Leakage**, the core diagnostic signal in this release, refers to a model being steered by spurious or hidden causal cues embedded in its inputs or context — rather than reasoning transparently from stated evidence. Think of it as the AI equivalent of a human expert whose judgment is unconsciously biased by irrelevant contextual signals they can't articulate. The framework's diagnostic approach works roughly as follows: Importantly, VIDRAFT is explicit that detecting an anomalous signal in the Causal Leakage dimension does **not** mean the model has caused harm or compromised any system. It indicates a *potential vulnerability* that warrants further investigation — closer to a security scanner flagging a suspicious pattern than a confirmed exploit. The source article does not publish quantitative scores or percentile rankings, but provides these qualitative findings: VIDRAFT plans to expand both the number of models evaluated and the scope of the diagnostic dataset over time as the leaderboard matures. The AX-RAY leaderboard and evaluation dataset are **publicly available on Hugging Face**. According to VIDRAFT's announcement, developers can access them directly via the Hugging Face platform. To browse or download the dataset using the Hugging Face CLI: ``` pip install huggingface_hub huggingface-cli login # Then search for VIDRAFT's AX-RAY dataset on huggingface.co/vidraft ``` Note:Specific dataset repository slugs and leaderboard URLs were not published in the source article. Visit[huggingface.co]and search forVIDRAFTorAX-RAYto locate the official dataset and leaderboard space. No API endpoint, model weights, or SDK beyond the dataset have been announced as part of this release. **Q: Is Causal Leakage the same as prompt injection or jailbreaking?** A: Not exactly. Prompt injection involves deliberate adversarial manipulation of a model's input. Causal Leakage is a subtler, often unintentional phenomenon — the model's internal reasoning is swayed by hidden or spurious causal signals in its context, even without an adversary deliberately engineering them. It's more of a structural reasoning flaw than an attack vector, though it could potentially be exploited. **Q: Does the leaderboard only evaluate safety, or does it also cover capability?** A: Based on the source, AX-RAY is exclusively a **safety diagnostic framework**. It is not designed to benchmark performance on reasoning, coding, or knowledge tasks. Its 117 evaluation items specifically target latent risk behaviors, potential norm violations, and regulatory non-compliance — making it complementary to, not a replacement for, capability benchmarks like MMLU or HumanEval. **Q: Why does AX-RAY include jurisdiction-specific and cultural normative frameworks?** A: AI safety is not culturally neutral. A model deployed in a financial advisory context in one country may face entirely different legal obligations than the same model deployed in a healthcare context elsewhere. By mapping diagnostic results to local laws and — in some cases — religious or social normative systems, AX-RAY aims to make safety evaluations actionable for teams building region-specific or regulated-industry applications. **Q: Does being flagged on AX-RAY mean a model is unsafe to use?** A: No. VIDRAFT is clear that a flagged Causal Leakage signal represents a *potential vulnerability* identified in evaluation scenarios — not evidence that the model has produced harmful outputs or security breaches in production. Teams should treat it as a signal for deeper investigation in their specific deployment context. *Originally reported by IT조선 (2026-08-18) — source article.*