A structurally chunked, pre-embedded SQLite corpus of the EU AI Act

Researchers released a structurally chunked, pre-embedded SQLite corpus of the EU AI Act, containing 933 chunks with BGE-M3 dense embeddings for retrieval-augmented generation. The corpus is designed for legal AI research and engineering, not as authoritative legal interpretation.

Paper • 2603.09435 • Published The dataset viewer is not available because its heuristics could not detect any supported data files https://huggingface.co/docs/hub/datasets-adding file-formats . You can try uploading https://huggingface.co/docs/hub/datasets-adding some data files, or configuring https://huggingface.co/docs/hub/datasets-data-files-configuration the data files location manually. EU AI Act — structural RAG corpus A single-file, pre-embedded SQLite corpus of the EU AI Act — Regulation EU 2024/1689 — chunked on legal structure only : one chunk per article paragraph, per recital, per annex point, per Article 3 definition. Chapter, section, and article metadata live in columns, not smeared into the text. The artifact is the corpus, not a pipeline — a downloadable database file you query locally, not a hosted service and not an authoritative legal interpretation. Release | Release | v1.0.0 prepared 2026-07-16 | | Primary artifact | aiact openrag.db SQLite, embeddings included | | Schema version | 2 recorded in the meta table | | Chunks | 933 | | Source text | EUR-Lex consolidated version, CELEX 02024R1689-20240712 | | Source commit | f94a363 | | Label audit commit | 03e7aff | Licensing is mixed — no single licence applies indiscriminately to every file.The EU legal text is reused under Commission Decision 2011/833/EU and isnot relicensedby this dataset; the original corpus structure, metadata, embeddings and documentation are CC BY 4.0; adapted benchmark data is CC BY 4.0; software associated with the source repository is Apache-2.0. See LICENSE for the path-level division LICENSES.md is the same text under the repository's canonical filename and NOTICE for the attribution notice. This is not legal advice.This corpus is a retrieval artifact for research and engineering. The chunking, tier labels, and metadata are operator-derived with documented rules — they arenot authoritative interpretations. In particular, risk tier is a labelling convention derived from the Act's structure see docs/derivation.md , not a legal determination. Consult the Official Journal text and qualified counsel for compliance decisions. Contents 933 chunks : 180 recitals, 522 article-paragraph chunks, 68 Article 3 definitions, 163 annex points BGE-M3 dense embeddings 1024-dim float32, L2-normalized for every chunk risk tier / obligation on labels derived only where the text is unambiguous — every rule cites its provision in; NULL rather than guess docs/derivation.md - ELI deep links to the exact provision on EUR-Lex - Staged application dates per Article 113 Source: EUR-Lex consolidated version 02024R1689-20240712 English . The English consolidated text is character-identical to the OJ text 32024R1689 ; article numbering is identical in both, and both CELEX ids are recorded. The build verifies, mechanically and fatally: every article 1–113 appears exactly once; every text node of the normative text is consumed by exactly one chunk and survives verbatim into the output zero silent drops ; no empty chunks; Annex III point counts match an independent extraction. Schema — table EmbeddingContent | column | type | notes | |---|---|---| chunk id | INTEGER PRIMARY KEY | | celex id | TEXT | 02024R1689-20240712 | content type | TEXT | recital | article | annex | definition | chapter | TEXT | e.g. CHAPTER III — HIGH-RISK AI SYSTEMS | section | TEXT | e.g. SECTION 2 — Requirements for high-risk AI systems | article num | TEXT | 6 , 6 2 , 3 34 , recital 44 , Annex III.4 a | heading | TEXT | e.g. Article 6 — Classification rules for high-risk AI systems | heading level | INTEGER | 1 recital, 3 article/annex intro, 4 paragraph/point, 5 sub-point | chunk text | TEXT | the provision text, nothing else | risk tier | TEXT | direct textual classification only: prohibited Art 5 1 | high Arts 6 1 /6 2 + Annex III, rebuttable per Art 6 3 | NULL. limited / minimal are reserved, never assigned — operator-derived, see disclaimer | regime bucket | TEXT | regime association: prohibited-practices | high-risk | transparency | gpai | voluntary-codes | NULL | obligation on | TEXT | chunk-level operative subject: provider | deployer | importer | distributor | NULL | eli url | TEXT | deep link to the exact provision | date in force | TEXT | EARLIEST date the provision applies, per Art 113 staging Annex I: NULL — an application date, not the Regulation's entry-into-force date 1 August 2024 | embedding | BLOB | 1024 × float32, raw bytes, L2-normalized BGE-M3 | continued | INTEGER | 0 normally; 1,2,… for parts of a paragraph split at ~1000 tokens | A meta table records the source URL, both CELEX ids, embedding model, source-text licence notice, schema version = 2 , and a semantics note for each of the four derived columns risk tier semantics , regime bucket semantics , obligation on semantics , date in force semantics stating exactly what each column does and does not claim. The four derived columns answer four different questions: risk tier — does this chunk's own operative text classify? Narrow and deliberately sparse: only direct textual classifications "shall be prohibited", "shall be considered to be high-risk" . regime bucket — which regulatory regime does this provision belong to? Structural association chapter/annex membership and the Commission's conventional tier names , independent of whether the text classifies. obligation on — which enum role is the chunk's operative obligated subject? Judged per chunk from its own sentences; NULL for authority-, Commission-, or AI-Office-bound and descriptive chunks. date in force — earliest application date per Article 113; not the entry-into-force date, and NULL where no single date is unambiguous. Value counts 933 chunks : | column | population | |---|---| risk tier | prohibited 2 · high 35 · NULL 896 | regime bucket | prohibited-practices 10 · high-risk 332 · transparency 7 · gpai 51 · voluntary-codes 4 · NULL 529 | obligation on | provider 56 · deployer 17 · importer 7 · distributor 6 · NULL 847 | Decoding the embeddings python import sqlite3, numpy as np con = sqlite3.connect "aiact openrag.db" con.row factory = sqlite3.Row row = con.execute "SELECT FROM EmbeddingContent WHERE article num = '5 1 '" .fetchone vec = np.frombuffer row "embedding" , dtype=np.float32 shape 1024, Getting started — semantic search in ~15 lines Encoding new queries needs the embedding model pip install sentence-transformers ; everything else is the standard library plus NumPy. python import sqlite3, numpy as np from sentence transformers import SentenceTransformer con = sqlite3.connect "aiact openrag.db" rows = con.execute "SELECT chunk id, article num, heading, chunk text, embedding " "FROM EmbeddingContent WHERE embedding IS NOT NULL" .fetchall mat = np.vstack np.frombuffer r 4 , dtype=np.float32 for r in rows model = SentenceTransformer "BAAI/bge-m3" q = model.encode "is emotion recognition at work prohibited?" , normalize embeddings=True 0 for i in np.argsort - mat @ q :5 : r = rows i print f"{r 1 :<16} {r 2 }\n {r 3 :150 }…\n" Filter by metadata instead of or as well as similarity — that is the point of structural chunking: -- what the text itself classifies direct, defensible per provision SELECT article num, chunk text FROM EmbeddingContent WHERE risk tier = 'prohibited'; -- everything belonging to a regime the broader, structural grouping SELECT article num, heading FROM EmbeddingContent WHERE regime bucket = 'high-risk' AND content type = 'article'; Evaluation The corpus was evaluated against the AI Act Evaluation Benchmark Davvetas et al. on retrieval, QA and risk-classification tasks; full numbers, methodology and provenance are in eval/RESULTS.md /datasets/faitholopade/aiact-openrag/blob/main/eval/RESULTS.md . One finding deserves a plain-language caveat here. A three-family model panel mistral-nemo, qwen2.5:14b, claude-fable-5 rating all 339 benchmark scenarios showed inter-model agreement collapsing exactly on the two tiers where classification F1 is lowest: per-tier Fleiss κ was 0.238 limited and 0.449 minimal versus 0.784 prohibited and 0.738 high-risk . This is consistent with ambiguity or noise at the limited/minimal label boundary , and it means limited/minimal per-tier scores on this benchmark should not be read as pure retrieval-quality signals — but it is an exploratory finding with substantial caveats , not proof: the raters are themselves LLMs, so shared model difficulty on a genuinely hard boundary cannot be separated from label noise, and the benchmark's labels are LLM-generation targets under human-authored tier prompts rather than independently validated classifications. Known limitations Plain language, so nobody trips on these later: Frozen in time. This is the consolidated version of the Act as of its consolidation date CELEX 02024R1689-20240712 , as fetched from EUR-Lex at build time July 2026 . If the Act is amended after that, this corpus does not know. Check EUR-Lex for anything load-bearing. Harmonised standards are not in here — on purpose. The CEN/CENELEC standards that operationalise the Act are copyrighted and cannot be redistributed. Commission guidance without an explicit reuse notice is also excluded. A RAG system built on this corpus alone cannot answer "which standard do I follow?"The Regulation's operative text never classifies a system into those tiers; they exist only in the Commission's four-tier presentation. An empty result for risk tier values limited and minimal never occur — on purpose. risk tier = 'minimal' is correct behaviour, not missing data. The conventional associations are preserved in regime bucket transparency ≈ "limited risk", Art 50; voluntary-codes ≈ "minimal risk", Art 95 .Only 37 of 933 chunks carry it 2 prohibited, 35 high because almost none of the Act's text directly classifies anything — most of it is regime machinery. Use risk tier is deliberately sparse. regime bucket 404 non-NULL for "provisions of the high-risk/prohibited/GPAI regimes". All derived columns are rule-derived, not court-decided. Every rule and the provision it rests on is written out in, and the audit that produced the current design in docs/derivation.md . Where the text is ambiguous the value is NULL. Read the rules before trusting the labels. docs/label-audit.md Attribution Source text : © European Union, 1998–2026, via EUR-Lex https://eur-lex.europa.eu/eli/reg/2024/1689/oj . Reuse permitted under Commission Decision 2011/833/EU https://eur-lex.europa.eu/eli/dec/2011/833/oj on the reuse of Commission documents; reproduction with acknowledgement of the source. Only the versions of EU law published in the Official Journal of the European Union are authentic. Evaluation benchmark : scenarios and QA pairs from the AI Act Evaluation Benchmark https://github.com/davidath/ai-act-evaluation-benchmark CC-BY-4.0 data, Apache-2.0 code . Citation If you evaluate against the benchmark used in this corpus's evaluation, cite: @article{Davvetas2026, title = {AI Act Evaluation Benchmark: An Open, Transparent, and Reproducible Evaluation Dataset for NLP and RAG Systems}, author = {Athanasios Davvetas and Michael Papademas and Xenia Ziouvelou and Vangelis Karkaletsis}, journal = {arXiv preprint arXiv:2603.09435}, year = {2026} } - Downloads last month - -