# A structurally chunked, pre-embedded SQLite corpus of the EU AI Act

> Source: <https://huggingface.co/datasets/faitholopade/aiact-openrag>
> Published: 2026-07-17 08:09:39+00:00

Paper • 2603.09435 • Published

The dataset viewer is not available because its heuristics could not detect any [supported data files](https://huggingface.co/docs/hub/datasets-adding#file-formats). You can try [uploading](https://huggingface.co/docs/hub/datasets-adding) some data
files, or [configuring](https://huggingface.co/docs/hub/datasets-data-files-configuration) the data
files location manually.

# EU AI Act — structural RAG corpus

A single-file, pre-embedded SQLite corpus of the **EU AI Act** — Regulation
(EU) 2024/1689 — chunked on **legal structure only**: one chunk per article
paragraph, per recital, per annex point, per Article 3 definition. Chapter,
section, and article metadata live in columns, not smeared into the text.
**The artifact is the corpus, not a pipeline** — a downloadable database
file you query locally, not a hosted service and not an authoritative
legal interpretation.

## Release

| Release | v1.0.0 (prepared 2026-07-16) |
| Primary artifact | `aiact_openrag.db` (SQLite, embeddings included) |
| Schema version | 2 (recorded in the `meta` table) |
| Chunks | 933 |
| Source text | EUR-Lex consolidated version, CELEX 02024R1689-20240712 |
| Source commit | `f94a363` |
| Label audit commit | `03e7aff` (
|

Licensing is mixed — no single licence applies indiscriminately to every file.The EU legal text is reused under Commission Decision 2011/833/EU and isnot relicensedby this dataset; the original corpus structure, metadata, embeddings and documentation are CC BY 4.0; adapted benchmark data is CC BY 4.0; software associated with the source repository is Apache-2.0. See[LICENSE]for the path-level division ([LICENSES.md]is the same text under the repository's canonical filename) and[NOTICE]for the attribution notice.

This is not legal advice.This corpus is a retrieval artifact for research and engineering. The chunking, tier labels, and metadata are operator-derived with documented rules — they arenot authoritative interpretations. In particular,`risk_tier`

is a labelling convention derived from the Act's structure (see`docs/derivation.md`

), not a legal determination. Consult the Official Journal text and qualified counsel for compliance decisions.

## Contents

**933 chunks**: 180 recitals, 522 article-paragraph chunks, 68 Article 3 definitions, 163 annex points** BGE-M3 dense embeddings**(1024-dim float32, L2-normalized) for every chunk`risk_tier`

/`obligation_on`

labels derived**only where the text is unambiguous**— every rule cites its provision in; NULL rather than guess`docs/derivation.md`

- ELI deep links to the exact provision on EUR-Lex
- Staged application dates per Article 113

Source: EUR-Lex consolidated version **02024R1689-20240712** (English). The
English consolidated text is character-identical to the OJ text
(`32024R1689`

); article numbering is identical in both, and both CELEX ids
are recorded. The build verifies, mechanically and fatally: every article
1–113 appears exactly once; every text node of the normative text is consumed
by exactly one chunk and survives verbatim into the output (zero silent
drops); no empty chunks; Annex III point counts match an independent
extraction.

## Schema — table `EmbeddingContent`

| column | type | notes |
|---|---|---|
`chunk_id` |
INTEGER PRIMARY KEY | |
`celex_id` |
TEXT | `02024R1689-20240712` |
`content_type` |
TEXT | `recital` | `article` | `annex` | `definition` |
`chapter` |
TEXT | e.g. `CHAPTER III — HIGH-RISK AI SYSTEMS` |
`section` |
TEXT | e.g. `SECTION 2 — Requirements for high-risk AI systems` |
`article_num` |
TEXT | `6` , `6(2)` , `3(34)` , `recital 44` , `Annex III.4(a)` |
`heading` |
TEXT | e.g. `Article 6 — Classification rules for high-risk AI systems` |
`heading_level` |
INTEGER | 1 recital, 3 article/annex intro, 4 paragraph/point, 5 sub-point |
`chunk_text` |
TEXT | the provision text, nothing else |
`risk_tier` |
TEXT | direct textual classification only: `prohibited` (Art 5(1)) | `high` (Arts 6(1)/6(2) + Annex III, rebuttable per Art 6(3)) | NULL. `limited` /`minimal` are reserved, never assigned — operator-derived, see disclaimer |
`regime_bucket` |
TEXT | regime association: `prohibited-practices` | `high-risk` | `transparency` | `gpai` | `voluntary-codes` | NULL |
`obligation_on` |
TEXT | chunk-level operative subject: `provider` | `deployer` | `importer` | `distributor` | NULL |
`eli_url` |
TEXT | deep link to the exact provision |
`date_in_force` |
TEXT | EARLIEST date the provision applies, per Art 113 staging (Annex I: NULL) — an application date, not the Regulation's entry-into-force date (1 August 2024) |
`embedding` |
BLOB | 1024 × float32, raw bytes, L2-normalized (BGE-M3) |
`continued` |
INTEGER | 0 normally; 1,2,… for parts of a paragraph split at ~1000 tokens |

A `meta`

table records the source URL, both CELEX ids, embedding model,
source-text licence notice, `schema_version = 2`

, and a `*_semantics`

note
for each of the four derived columns (`risk_tier_semantics`

,
`regime_bucket_semantics`

, `obligation_on_semantics`

,
`date_in_force_semantics`

) stating exactly what each column does and does
not claim.

The four derived columns answer four different questions:

`risk_tier`

—*does this chunk's own operative text classify?*Narrow and deliberately sparse: only direct textual classifications ("shall be prohibited", "shall be considered to be high-risk").`regime_bucket`

—*which regulatory regime does this provision belong to?*Structural association (chapter/annex membership and the Commission's conventional tier names), independent of whether the text classifies.`obligation_on`

—*which enum role is the chunk's operative obligated subject?*Judged per chunk from its own sentences; NULL for authority-, Commission-, or AI-Office-bound and descriptive chunks.`date_in_force`

—*earliest application date*per Article 113; not the entry-into-force date, and NULL where no single date is unambiguous.

Value counts (933 chunks):

| column | population |
|---|---|
`risk_tier` |
`prohibited` 2 · `high` 35 · NULL 896 |
`regime_bucket` |
`prohibited-practices` 10 · `high-risk` 332 · `transparency` 7 · `gpai` 51 · `voluntary-codes` 4 · NULL 529 |
`obligation_on` |
`provider` 56 · `deployer` 17 · `importer` 7 · `distributor` 6 · NULL 847 |

## Decoding the embeddings

``` python
import sqlite3, numpy as np

con = sqlite3.connect("aiact_openrag.db")
con.row_factory = sqlite3.Row
row = con.execute("SELECT * FROM EmbeddingContent WHERE article_num = '5(1)'").fetchone()
vec = np.frombuffer(row["embedding"], dtype=np.float32)   # shape (1024,)
```

## Getting started — semantic search in ~15 lines

Encoding new queries needs the embedding model (`pip install sentence-transformers`

); everything else is the standard library plus NumPy.

``` python
import sqlite3, numpy as np
from sentence_transformers import SentenceTransformer

con = sqlite3.connect("aiact_openrag.db")
rows = con.execute("SELECT chunk_id, article_num, heading, chunk_text, embedding "
                   "FROM EmbeddingContent WHERE embedding IS NOT NULL").fetchall()
mat = np.vstack([np.frombuffer(r[4], dtype=np.float32) for r in rows])

model = SentenceTransformer("BAAI/bge-m3")
q = model.encode(["is emotion recognition at work prohibited?"],
                 normalize_embeddings=True)[0]
for i in np.argsort(-(mat @ q))[:5]:
    r = rows[i]
    print(f"{r[1]:<16} {r[2]}\n   {r[3][:150]}…\n")
```

Filter by metadata instead of (or as well as) similarity — that is the point of structural chunking:

```
-- what the text itself classifies (direct, defensible per provision)
SELECT article_num, chunk_text FROM EmbeddingContent
WHERE risk_tier = 'prohibited';

-- everything belonging to a regime (the broader, structural grouping)
SELECT article_num, heading FROM EmbeddingContent
WHERE regime_bucket = 'high-risk' AND content_type = 'article';
```

## Evaluation

The corpus was evaluated against the AI Act Evaluation Benchmark
(Davvetas et al.) on retrieval, QA and risk-classification tasks; full
numbers, methodology and provenance are in
[eval/RESULTS.md](/datasets/faitholopade/aiact-openrag/blob/main/eval/RESULTS.md).

One finding deserves a plain-language caveat here. A three-family
model panel (mistral-nemo, qwen2.5:14b, claude-fable-5) rating all 339
benchmark scenarios showed inter-model agreement collapsing exactly on the
two tiers where classification F1 is lowest: per-tier Fleiss κ was 0.238
(limited) and 0.449 (minimal) versus 0.784 (prohibited) and 0.738
(high-risk). This is **consistent with ambiguity or noise at the
limited/minimal label boundary**, and it means limited/minimal per-tier
scores on this benchmark should not be read as pure retrieval-quality
signals — but it is an **exploratory finding with substantial caveats**,
not proof: the raters are themselves LLMs, so shared model difficulty on a
genuinely hard boundary cannot be separated from label noise, and the
benchmark's labels are LLM-generation targets under human-authored tier
prompts rather than independently validated classifications.

## Known limitations

Plain language, so nobody trips on these later:

**Frozen in time.** This is the consolidated version of the Act as of its consolidation date (CELEX`02024R1689-20240712`

), as fetched from EUR-Lex at build time (July 2026). If the Act is amended after that, this corpus does not know. Check EUR-Lex for anything load-bearing.**Harmonised standards are not in here — on purpose.** The CEN/CENELEC standards that operationalise the Act are copyrighted and cannot be redistributed. Commission guidance without an explicit reuse notice is also excluded. A RAG system built on this corpus alone cannot answer "which standard do I follow?"The Regulation's operative text never classifies a system into those tiers; they exist only in the Commission's four-tier presentation. An empty result for`risk_tier`

values`limited`

and`minimal`

never occur — on purpose.`risk_tier = 'minimal'`

is correct behaviour, not missing data. The conventional associations are preserved in`regime_bucket`

(`transparency`

≈ "limited risk", Art 50;`voluntary-codes`

≈ "minimal risk", Art 95).Only 37 of 933 chunks carry it (2 prohibited, 35 high) because almost none of the Act's text directly classifies anything — most of it is regime machinery. Use`risk_tier`

is deliberately sparse.`regime_bucket`

(404 non-NULL) for "provisions of the high-risk/prohibited/GPAI regimes".**All derived columns are rule-derived, not court-decided.** Every rule and the provision it rests on is written out in, and the audit that produced the current design in`docs/derivation.md`

. Where the text is ambiguous the value is NULL. Read the rules before trusting the labels.`docs/label-audit.md`

## Attribution

**Source text**: © European Union, 1998–2026, via[EUR-Lex](https://eur-lex.europa.eu/eli/reg/2024/1689/oj). Reuse permitted under[Commission Decision 2011/833/EU](https://eur-lex.europa.eu/eli/dec/2011/833/oj)on the reuse of Commission documents; reproduction with acknowledgement of the source. Only the versions of EU law published in the Official Journal of the European Union are authentic.**Evaluation benchmark**: scenarios and QA pairs from the[AI Act Evaluation Benchmark](https://github.com/davidath/ai-act-evaluation-benchmark)(CC-BY-4.0 data, Apache-2.0 code).

## Citation

If you evaluate against the benchmark used in this corpus's evaluation, cite:

```
@article{Davvetas2026,
  title   = {AI Act Evaluation Benchmark: An Open, Transparent, and
             Reproducible Evaluation Dataset for NLP and RAG Systems},
  author  = {Athanasios Davvetas and Michael Papademas and Xenia Ziouvelou
             and Vangelis Karkaletsis},
  journal = {arXiv preprint arXiv:2603.09435},
  year    = {2026}
}
```

- Downloads last month
- -