🧠 I Trained a Massive Word2Vec Model on 13 Billion Russian Fiction Words — Here’s What Happened

TL;DR: I built a lemma‑based Word2Vec model (CBOW, 300d) on a huge corpus of Russian fiction (13B tokens → 7.3B after cleaning). You can load it with Gensim and explore semantic neighborhoods of words like слово, язык, речь. The model captures literary semantics without stop words or grammar tags. Check it out on Hugging Face.

Most pre‑trained Russian word2vec models are trained on web crawls, news, or mixed corpora. That’s fine for general NLP, but fiction has its own semantic rules. Poetic metaphors, archaic vocabulary, and author‑specific styles shift vector spaces.

I wanted a model that:

So I built one. And I’m sharing it under the MIT license.

Stop words: removed using this list.

Lemmatizer: Yandex Mystem.

Sentence splitter: razdel.sentenize.

All lemmas are lowercase, dictionary form, no part‑of‑speech tags (saves time & space).

import gensim

data = gensim.models.word2vec.LineSentence('splitted_lemmed_lines.txt')
model = gensim.models.Word2Vec(
    data,
    vector_size=300,
    window=10,
    min_count=2,
    sg=0          # CBOW (faster, good for large corpora)
)
model.save('cbow_300_10.model')

The training script is plain Gensim – no weird dependencies. You can retrain or fine‑tune if you have more data.

When you clone from Hugging Face, you get:

cbow_300_10.model (197 MB) – the main Gensim model object cbow_300_10.model.syn1neg.npy (5.8 GB) – negative sampling weights cbow_300_10.model.wv.vectors.npy (5.8 GB) – the actual word vectors README.md, tst.py – docs and a test script Yes, the two .npy files are large. But you can load the model without both if you only need similarities (Gensim does lazy ). Or use model.wv directly.

import gensim

model = gensim.models.Word2Vec.load("cbow_300_10.model")

for word, score in model.wv.most_similar("слово", topn=10):
    print(f"{word}: {score:.4f}")

Output:

фраза: 0.7941
словечко: 0.6602
слог: 0.6322
реплика: 0.6015
словосочетание: 0.5928
изречение: 0.5818
высказывание: 0.5800
глагол: 0.5735
эпитет: 0.5615
сентенция: 0.5556

Notice how epithet and verb pop up – the model clearly learned linguistic meta‑concepts from fiction.

Now try язык (language / tongue):

наречие (adverb) 0.6684
диалект 0.6095
латынь 0.5892
язычок (little tongue) 0.5698
алфавит 0.5039
грамматика 0.5027
суахили 0.4977
идиома 0.4952
иврит 0.4950
произношение 0.4927

And речь (speech):

монолог 0.6400
спич 0.5914
тирада 0.5900
фраза 0.5652
диалог 0.5477
проповедь 0.5334
слово 0.5217
разглагольствование 0.5193
филиппика 0.5184
декламация 0.5147

стекло could be noun “glass” or past tense of “to flow”). That’s a conscious trade‑off for performance.слова, слову, словом? You won’t find them; use the lemma слово. KeyedVectors.load() to save RAM.

If you need a general‑purpose model with grammatical info, go for RusVectores. If you work with literary analysis, this one is your friend.

pip install gensim and model = gensim.models.Word2Vec.load(...) .npy files are optional. Hugging Face: nevmenandr/w2v-russian-fiction

License: MIT

Clone with git lfs or download via huggingface_hub:

from huggingface_hub import snapshot_download
snapshot_download(repo_id="nevmenandr/w2v-russian-fiction", local_dir="./w2v_model")

model.most_similar(positive=['царь', 'женщина'], negative=['мужчина']) and share the result. (Hint: it’s not “царица” – fiction is weird.)

import gensim

model = gensim.models.Word2Vec.load("cbow_300_10.model")

print("Neighbors of поэт:")
for w, s in model.wv.most_similar("поэт", topn=5):
    print(f"  {w}: {s:.4f}")

result = model.wv.most_similar(positive=["Франция", "Москва"], negative=["Россия"], topn=1)
print(f"\nМосква / Россия ≈ Париж / {result[0][0]} (score {result[0][1]:.4f})")

Run it. Play with it. Break it. Then tell me in the comments what you found.

Happy vector hunting! 🧙♂️

P.S. The model is called cbow_300_10.model – old‑school name, but it works like a charm.