Show HN: Anime Text Datasets for LLMs Training

Text-only anime dataset built for fine-tuning large language models (LLM), instruction tuning, retrieval-augmented generation (RAG), question answering (QA), entity linking, recommendation systems, and evaluation benchmarks. The headline unit is unique anime works — one work expands into many records: characters, staff, episodes, QA pairs, RAG chunks, and relationship edges.

Developer & maintainer: Muhammad Muqeet — contact@animearray.site · muqeet@animearray.site · muqeet@anitroves.com

Live: dataset site · Hugging Face Space · full release · free 30% for LLM training · CC0 subset

Metric	Value
Unique anime works with at least one title	35,766
Characters	283,477
Staff credits	535,642
Episodes	276,763
Alternative titles	135,425
Relationship edges	39,226
Franchise clusters	23,980
Descriptions (summaries + synopses)	63,396
Question/answer pairs	317,907
Instruction-tuning examples	235,395
RAG chunks	206,906
Split: train / val / test	32,189 / 1,788 / 1,789

Coverage of the anime table (share of rows carrying a value):

Field	Coverage
Year	95%
Episode count	94%
Per-episode records	60%
Studios	63%
Genres	77%
Characters	50%
Voice actors	33%
Staff credits	65%
Relationships	50%

Every export ships with SHA-256 checksums, and python -m src.cli validate runs 21 checks covering uniqueness, referential integrity, enum conformance, value ranges, split hygiene, answer grounding, corpus diversity, license coverage and export schema.

The tables above are generated from reports/summary.json by python scripts/sync_docs.py; they are never edited by hand. The site pages are written in docs_src/ and rendered into docs/ by the same script, which keeps the {{n:…}} count tokens in the source so later runs can refresh them.

A structured, factual, text-only corpus about anime: the works themselves, their casts, the people who made them, the studios, the genres and themes, the episode lists, the narrative relations between works — and, most importantly for NLP, hundreds of thousands of grounded rows of natural language about anime.

Each anime work includes:

• Identity and titles — canonical, English, Japanese, and up to dozens of alternative titles per work.
• Discovery metadata — format, status, episode count, duration, air dates, season bucket, year.
• Text content — a generated record summary built from the work's own structured fields, plus the upstream synopsis where one was published, carryingprose_source andprose_license (seeLICENSE_AUDIT.md ).
• Scores and popularity — mean score normalised to a single 0–100 scale, popularity, favourites.
• Taxonomy — genre and theme labels, source material, age rating.
• Cast and crew — characters with roles and voice actors, staff with per-record credit roles.
• Episodes — per-episode numbers, titles, air dates and lengths.
• Relations — prequel/sequel/side-story/spin-off edges plus franchise clusters.
• Training-ready text — QA pairs, instruction examples and RAG chunks, each grounded in fields of the same record.
• Provenance and a license tier on every record — source, remote ID, link method, confidence, and one of four redistribution tiers.

Collection	Rows per anime	What it teaches
qa.jsonl	6–14	Direct question answering over one work: metadata, cast, crew, episodes, relations, franchise, multi-hop, comparison, synopsis
instructions.jsonl	0–12	Task formats: extraction, summarisation, JSON repair, title normalisation, entity resolution, abstention
rag_chunks.jsonl	4–10	Retrieval units written as prose, each with source_refs and akind label
descriptions.jsonl	2	record_summary generated from structured fields, plussource_synopsis with its source

QA pairs are emitted only when every value in the answer exists in the database for that work. Each question type has several phrasings chosen deterministically per work, so the corpus is not one repeated template. Roughly one pair in twenty is an explicit abstention ("No episode count is recorded for this work in this dataset") so a model can learn to decline instead of inventing.

Instruction examples are derivable from their own input field by construction: the source facts are rendered into the input, then the task asks for a transformation of exactly those facts. This is checked at validation time by re-reading the input and confirming each output value appears in it.

RAG chunks are prose, not JSON dumps, and are labelled by family (record_summary, source_synopsis, title_aliases, characters, voice_cast, staff, episodes, relations). source_synopsis chunks are upstream text and carry prose_source + prose_license.

• No images, posters, audio, subtitles, or lyrics.
• No full scripts or long copyrighted passages.
• No human-written question/answer pairs: the QA and instruction corpora are template-generated from structured fields. They are supervision for extraction and grounding, not a source of new facts.
• Not uniformly licensed: retained synopses, biographies, episode summaries and tag definitions are third-party prose, excluded from the project's CC-BY-4.0 grant. See LICENSE_AUDIT.md .

These are measured, not estimated — the exact figures are regenerated into reports/final_report.md on every run.

• Episode lists are the weakest field. Kitsu is the only reachable source that publishes per-episode records. Shikimori's episode endpoint is unavailable, AniList publishes counts but not episode lists, and MyAnimeList/Jikan is network-blocked from the collection environment. Works without a Kitsu link therefore have an episodecount but no episodelist .
• Depth is uneven by design. Characters, staff and episodes exist only where an upstream source published them. The coverage table above gives the exact rate per field instead of an average that hides the spread.
• The Anime News Network encyclopedia contributes zero rows. It is reachable only page-by-page, so a sample was collected and retained in raw storage without being linked into the exports.
• The 50,000-title target is not met. The union of the reachable catalogues is exhausted at the current count; MAL would raise it and is blocked.
• Franchise clusters use strong relations only (SEQUEL ,PREQUEL ,PARENT_STORY ,FULL_STORY ,SIDE_STORY ,SPIN_OFF ,ALTERNATIVE_SETTING ,ALTERNATIVE_VERSION ,SUMMARY ). Sharing a source manga or a cameo does not merge two works into one cluster.

Release	Works	Rows	Who can use it	How to get it
Full release	35,766	2,436,445	research and education	request access — gated, manual approval
30% preview	10,729	713,238	anyone with a Hugging Face account	get free access — gated, automatic approval
CC0 subset	969	8,654	anyone, no account	direct download — public
Code, schemas, docs	—	—	anyone	GitHub

• No purchase is required for any release. The 30% preview is free and approved instantly. The full release is approved manually for non-commercial research or education.
• Gating is access control, not a licence. The full release and the preview contain third-party metadata and retained upstream prose. Approval does not grant redistribution rights from AniList, Kitsu, Shikimori or Anime News Network.
• Commercial use needs permission from the relevant upstream source, except for thecc0_only rows (969 works).
• Redistribution is permitted only for the public CC0 subset. Do not mirror or republish the full release or the preview.

muqeet-1138/anime-text-dataset-preview is a free, automatically approved 30% sample of the full release. It exists so people can evaluate the data and train models without buying anything or requesting the full dataset.

Property	Value
Works	10,729 of 35,766 (30.0%)
Rows	713,238
Sampling	stratified by (license_tier, split) , seed 42
Child rows	every row of each sampled work is kept
Tiers	291 cc0_only / 3,398permission_required / 7,040unresolved
Splits	9,656 train / 536 val / 537 test
Formats	JSONL, Parquet, CSV (no SQLite or dedup log)
Access	free, gated, automatic approval
Training	allowed for non-commercial research or education

The sample is representative, not cherry-picked: every licence tier and split is preserved, so a model trained on it sees the same obligation mix as the full release. It is not a public-domain release and not a licence from the source owners. Only 291 of its works are cc0_only; the rest carry upstream obligations.

from datasets import load_dataset

qa = load_dataset("muqeet-1138/anime-text-dataset-preview", "qa", split="train")
anime = load_dataset("muqeet-1138/anime-text-dataset-preview", "anime", split="train")

Nine ready-to-use gateways into the same data:

#	Method	What you get	How
1	Clone the repository	code, schemas, docs, pipeline	git clone https://github.com/muqeet-1138/anime-text-dataset.git
2	Download the full data	the complete exports/ tree, 2.2 GB	request access on the gated Hugging Face dataset
3	Free 30% preview	10,729 works, 713,238 rows, instant approval	get free access
4	**Hugging Face datasets **	a sharded HF Dataset	load_dataset("muqeet-1138/anime-text-dataset-preview", "qa", split="train")
5	Stream JSONL in Python	train-ready rows without everything	loop over open("exports/qa.jsonl", encoding="utf-8")
6	SQLite database	joins and referential queries	sqlite3 exports/anime_dataset.sqlite or Python stdlib
7	Parquet + DuckDB / pandas	fast analytics	duckdb -c "select ..." orpd.read_parquet
8	CSV	spreadsheet-friendly flat view	open exports/anime.csv
9	Training recipes	fine-tuned models	follow docs/TRAINING.md +examples/ scripts

Each gateway is covered with runnable code in docs/COMMANDS.md (terminals) and docs/TRAINING.md (model training).

Primary format is JSONL (UTF-8, one logical record per line, deterministic field order, stable IDs). Derived formats are regenerated from the same canonical data.

Format	File	Notes
JSONL	exports/*.jsonl	canonical, 19 tables
SQLite	exports/anime_dataset.sqlite	all tables, split andlicense_tier pre-joined, indexed
Parquet	exports/*.parquet	10 tables, for analytics
CSV	exports/anime.csv	flat anime table
License subsets	exports/license_tier_*.csv	pre-filtered anime id lists per tier
Checksums	exports/checksums.json + per-file.sha256	verify integrity

Every per-anime table carries split and license_tier as columns, so a training script can filter without joining anything. JSON arrays are real arrays in JSONL and Parquet, not encoded strings.

Hosting note: GitHub holds the code, schemas and documentation (0.7 MB). The 2.2 GB generated dataset is hosted on Hugging Face, not in Git and not as GitHub Release assets. Three releases share one schema: the full mixed-rights release (gated, manual approval), a free 30% preview (gated, automatic approval, 10,729 works), and the public CC0 subset (ungated, 969 works). Every release publishes SHA-256 checksums. Contact: contact@animearray.site.

Verified checksum verification (any shell):

Get-ChildItem exports\*.jsonl | ForEach-Object {
  $h = (Get-FileHash $_.FullName -Algorithm SHA256).Hash.ToLower()
  if ($h -eq (Get-Content "$($_.FullName).sha256").Split(' ')[0]) { "OK  $($_.Name)" } else { "BAD $($_.Name)" }
}

Two ways to browse before training:

• Interactive explorer site —docs/explore.html renders the live statistical profile: coverage rates, year/type/status distributions, top genres, themes and studios, numeric quartiles for every length/score field, and the full export manifest.
• Static profile —reports/data_profile.json is the machine-readable version, regenerated any time withpython scripts/data_profile.py .
• EDA notebook —examples/notebooks/EDA.ipynb walks the same numbers in Jupyter with charts.
• Publish to Hugging Face Hub —python scripts/upload_hf.py --repo muqeet-1138/anime-text-dataset --gated manual uploads the full release to a gated dataset repo.python scripts/build_preview_subset.py thenpython scripts/upload_hf.py --repo muqeet-1138/anime-text-dataset-preview --source exports_preview --gated auto publishes the free 30% preview.python scripts/build_open_subset.py thenpython scripts/upload_hf.py --repo muqeet-1138/anime-text-dataset-open --source exports_open publishes the public CC0 subset. Run with--dry-run first; token viaHF_TOKEN or the login saved byhf auth login .

Source	Contributed	Terms	Tier assigned
AniList	discovery, scores, characters, staff, relations	public GraphQL API; terms allow API use, not redistribution of content	permission_required
Kitsu	discovery, episode lists	public JSON:API; terms grant no redistribution of API data	permission_required
Shikimori	discovery, cast, crew, relations	no license or terms for API data could be located	unresolved
Wikidata	supplementary titles	CC0-1.0	cc0_only
ANN encyclopedia	789 sample pages, retained in raw storage	copyrighted	not linked into exports
MyAnimeList / Jikan	none	—	unreachable, excluded

Each anime row is assigned exactly one tier: the strictest implied by the sources that contributed to it (unresolved > permission_required > cc0_only). The tier names describe the obligation you take on, not the copyright status of the underlying anime, which stays with its rights holders.

• cc0_only — no obligation; attribution is courtesy.
• permission_required — AniList and Kitsu terms do not grant redistribution; obtain written permission before publishing derived records.
• unresolved — no permission basis identified; do not redistribute.

Machine-readable: exports/license_tiers.jsonl (per anime), exports/license_manifest.json (definitions, counts, gaps, disclaimer), exports/sources.jsonl (per source), exports/provenance.jsonl (per record). Human-readable: docs/LICENSE_NOTES.md, NOTICE, ACKNOWLEDGMENTS.md.

The repository code, pipeline and project-generated text are released under CC-BY-4.0. That license covers the code and the text this project generates; it cannot and does not relicense upstream source content.

Upstream prose is retained, attributed, and excluded from that grant. Plot synopses, character biographies, episode summaries and tag definitions are third-party expression. Every prose row carries prose_source and prose_license; per-source owners and terms are in exports/attribution.json and the prose_attribution block of exports/license_manifest.json. Attribution is not permission. Every artifact and its rights status is listed in LICENSE_AUDIT.md.

Object	Export file	Content
Anime	anime.jsonl	canonical record per unique work, with nested titles/genres/themes/studios/sources and attributed synopsis
Characters	characters.jsonl	name, native name, localized name, role, voice actors, attributed biography
Staff	staff.jsonl	person, credit roles, source references
Studios	studios.jsonl	studio names + identifiers
Genres / themes	genres.jsonl ,themes.jsonl	controlled vocabulary; theme definitions attributed
Episodes	episodes.jsonl	number, title, air date, runtime, attributed summary
Relationships	relationships.jsonl	typed edges between works
Alternative titles	alt_titles.jsonl	language, kind, and the sources that supplied it
Descriptions	descriptions.jsonl	record_summary plus attributedsource_synopsis
QA pairs	qa.jsonl	question, answer, type, template, grounded fields, prose source
Instructions	instructions.jsonl	instruction, input, output, type, purpose
RAG chunks	rag_chunks.jsonl	prose chunk, kind ,source_refs , prose source
Entity links	entity_links.jsonl	internal → external IDs, method, confidence
Franchises	franchises.jsonl	anime → franchise cluster id
Provenance	provenance.jsonl +sources.jsonl +attribution.json	per-record, per-source, and per-prose-owner registry
License tiers	license_tiers.jsonl	per-anime redistribution tier and reason

Field	Type	Meaning
anime_id	int	stable internal ID
type	enum	TV ,MOVIE ,OVA ,ONA ,SPECIAL ,TV_SPECIAL ,SHORT ,TV_SHORT ,ONA_SHORT ,MUSIC ,PV ,CM
status	enum	FINISHED ,RELEASING ,NOT_YET_RELEASED ,CANCELLED ,HIATUS
episodes ,duration_min	int	episode count and per-episode minutes
aired_from ,aired_to	date	air window ( YYYY-MM-DD )
season ,year	enum/int	WINTER /SPRING /SUMMER /FALL and start year
title_canonical ,title_english ,title_japanese	string	titles
mean_score ,popularity ,favourites	number	score on a single normalised 0–100 scale
source_material ,age_rating	enum	e.g. MANGA ,PG-13
quality_tier	string	A / B / C
confidence	string	high/medium/low
synopsis ,synopsis_source	string	upstream synopsis and its registry id; prose_source /prose_license are added on export
alt_titles ,genres ,themes ,studios ,sources ,franchise_id ,license_tier ,split	list/string	enriched record

Full field-level reference: docs/DATA_DICTIONARY.md and docs/SCHEMAS.md.

• 90 / 5 / 5 (train / val / test), random seed42 .
• Split per anime , never per record — every character, QA pair, chunk, and relationship of a work lands in the same bucket, sono leakage across train/test.
• Every exported record carries its split so you can filter without re-walking the tables.

import json
train_qa = [q for q in map(json.loads, open("exports/qa.jsonl", encoding="utf-8")) if q["split"] == "train"]
val_qa   = [q for q in map(json.loads, open("exports/qa.jsonl", encoding="utf-8")) if q["split"] == "val"]
test_qa  = [q for q in map(json.loads, open("exports/qa.jsonl", encoding="utf-8")) if q["split"] == "test"]

Works identically from Windows CMD, PowerShell, and Git Bash / WSL. Replace python with python3 where your system uses it.

git clone <repo-url>
cd anime-text-dataset
python -m venv .venv
.venv\Scripts\activate        # Windows CMD/PowerShell
pip install -r requirements.txt
python -m src.cli validate       # run all 21 validation checks
python -m src.cli checksums      # regenerate SHA-256 checksums
python -m src.cli report         # regenerate reports/final_report.md + summary.json
python scripts/sync_docs.py      # refresh the generated tables in the docs

The full guide — SFT, LoRA/QLoRA, DPO, RAG fine-tuning, continual pretraining, embeddings, classifiers — lives in docs/TRAINING.md.

Command	Phase	Description
python -m src.cli discover	P0	collect payloads per source
python -m src.cli dedup	P1	entity resolution → canonical anime rows
python -m src.cli backfill	P1	fill missing fields from stored payloads
python -m src.cli refill	P1	re-fetch stale or failed payloads
python -m src.cli enrich	P2	full metadata for the depth tier
python -m src.cli entities	P3	materialize characters, staff, studios, episodes
python -m src.cli normalize	P3.5	merge duplicates, canonicalize enums, repair values, dedupe entities
python -m src.cli relationships	P4	derive relationship edges
python -m src.cli franchise	P5	cluster works into franchises
python -m src.cli descriptions	P6	write record summaries
python -m src.cli qa	P7	generate QA pairs
python -m src.cli instructions	P8	generate instruction-tuning examples
python -m src.cli rag	P9	build RAG chunks with source refs
python -m src.cli split	P10	assign train/val/test per anime
python scripts/license_tiers.py	P11	assign license tiers in the DB, write manifest and subsets
python -m src.cli export	P12	write JSONL/SQLite/Parquet/CSV with split +license_tier joined
python -m src.cli validate	QA	run all 21 validation checks
python -m src.cli checksums	QA	refresh SHA-256 checksums
python -m src.cli report	QA	regenerate the report
python scripts/sync_docs.py	QA	refresh generated doc tables from the report

Order matters in two places: license_tiers runs before export so the tier is a real column in the exported rows rather than a null, and validate runs after export because it checks the exported files.

Full flag and environment details: docs/COMMANDS.md.

(Get-Content exports\anime.jsonl | Measure-Object -Line).Lines
wc -l exports/anime.jsonl

Get-Content exports\anime.jsonl -TotalCount 1   # PowerShell

python -c "import json; A=[json.loads(l) for l in open('exports/anime.jsonl',encoding='utf-8')]; print(len(A), sum(1 for a in A if a['year']==2020))"

python -c "import sqlite3; con=sqlite3.connect('exports/anime_dataset.sqlite'); [print(r) for r in con.execute('select year,count(*) from anime group by year order by 2 desc limit 5')]"

duckdb -c "select year, count(*) n from read_parquet('exports/anime.parquet') group by year order by year desc limit 5;"

python -c "import csv; ids={r['anime_id'] for r in csv.DictReader(open('exports/license_commercial_clear.csv',encoding='utf-8'))}; print(len(ids), 'anime in the clear subset')"

Nine concrete training recipes:

#	Recipe	Output	Key files
1	Instruction fine-tuning (SFT)	chat-style assistant	qa.jsonl ,instructions.jsonl
2	Parameter-efficient fine-tuning (LoRA / QLoRA)	small-GPU fine-tune	instructions.jsonl
3	Preference tuning (DPO)	grounded responses	qa.jsonl
4	Retrieval-augmented generation (RAG) fine-tune	grounded QA	rag_chunks.jsonl ,qa.jsonl
5	Domain continual pretraining	anime-proficient base model	descriptions.jsonl
6	Contrastive / embedding training	semantic retrieval	rag_chunks.jsonl ,qa.jsonl
7	Classification & ranking heads	genre/source/rating predictors	anime.jsonl
8	Entity linking & relation extraction	KG construction	entity_links.jsonl ,relationships.jsonl
9	Evaluation & benchmarks	honest model reporting	all test-split files

Complete code, environment setup, hardware notes, and evaluation guidance: docs/TRAINING.md — the single authoritative guide.

import json, sqlite3
import pandas as pd

anime = [json.loads(l) for l in open("exports/anime.jsonl", encoding="utf-8")]
by_id = {a["anime_id"]: a for a in anime}

hits = [a for a in anime if "Cowboy" in (a.get("title_canonical") or "")]

con = sqlite3.connect("exports/anime_dataset.sqlite")
rows = con.execute(
    "select title_canonical, year from anime "
    "where year >= 2015 order by mean_score desc limit 10"
).fetchall()

df = pd.read_parquet("exports/anime.parquet")
df[df["year"] == 1998].head()

for line in open("exports/qa.jsonl", encoding="utf-8"):
    q = json.loads(line)
    if q["split"] == "train" and q["license_tier"] == "cc0_only":
        pass  # feed (q["question"], q["answer"]) to your trainer

Can I train an LLM on this? Yes — that is the primary use case. qa.jsonl and instructions.jsonl are ready-to-shard instruction formats; see docs/TRAINING.md.

Which part do I use for what? QA/instructions → chat fine-tuning; RAG chunks → retrieval; anime records → classification/ranking; relationships → knowledge-graph and relation extraction.

Is there leakage across train/test? No — splits are assigned per anime, so all records of one work stay in one bucket.

Can I publish a model trained on this? Check the license_tier and prose_license columns first. Only cc0_only rows have no upstream permission outstanding; permission_required and unresolved rows carry obligations that this project cannot discharge for you. This is an organisational aid, not legal advice.

Is the upstream prose licensed? No. Synopses, biographies, episode summaries and tag definitions are retained with a recorded source and are excluded from the CC-BY-4.0 grant. Attribution is not permission; check exports/attribution.json for the owner and terms of each source.

Why are some answers "not recorded"? About one QA pair in twenty is a deliberate abstention for a field that is genuinely absent, so a model can learn to decline rather than guess.

If you use this dataset — in research, a trained model, an evaluation, or an AI-generated answer — cite the release you used:

@misc{muqeet2026anime,
  author       = {Muhammad Muqeet},
  title        = {Large Anime Text Dataset},
  year         = {2026},
  version      = {2.1.0},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/datasets/muqeet-1138/anime-text-dataset},
  note         = {35,766 anime works; text-only; mixed-rights with a CC0-only subset}
}

• Full release: keep theurl above.
• 30% preview: usehttps://huggingface.co/datasets/muqeet-1138/anime-text-dataset-preview .
• CC0 subset: usehttps://huggingface.co/datasets/muqeet-1138/anime-text-dataset-open .

Machine-readable: CITATION.cff. Human guide: docs/CITATION.md. AI/LLM index: llms.txt.

If you retrieve, quote, summarize or train on this dataset, preserve the title, author, version and URL above. Do not describe the full release or the 30% preview as CC0 or public domain: both are mixed-rights and gated. Only the cc0_only tier is CC0 (969 works in the full release, 291 in the preview). When citing in prose, use:

Muhammad Muqeet. Large Anime Text Dataset, version 2.1.0. 2026. https://huggingface.co/datasets/muqeet-1138/anime-text-dataset

Contact for citation or access questions: contact@animearray.site.

The repository code, pipeline, and generated text are licensed under CC-BY-4.0. Upstream source content is not relicensed by that license: every anime row carries a license_tier derived from the metadata and prose sources that contributed to it, and exports/license_manifest.json states what each tier permits. Retained prose also carries prose_source and prose_license; owners and terms are in exports/attribution.json. The per-artifact audit is LICENSE_AUDIT.md.

Access and licensing questions: contact@animearray.site.

source & further reading

github.com — original article CasioEmuMsvc: Emulator and devkit for Casio nX-U8/100 and nX-U16/100 MCU series Show HN: AI agents trade privately on BitcoinZ, with proof of delivery Show HN: Korea-data-MCP – Korean public APIs that agents can't misread