{"slug": "annotating-genomes-via-unsupervised-deep-learning", "title": "Annotating Genomes via Unsupervised Deep Learning", "summary": "Vipsania, an unsupervised deep learning ab-initio gene finder from the Gaius-Augustus project, annotates eukaryotic genomes from a FASTA file alone without any reference annotation, curated gene set, or protein evidence. The tool trains purely on raw genomic sequences with a masked language modelling objective, requires python>=3.12, and outputs annotations in GenePred, GFF3, or GTF; it is installable via pip or pre-built Docker and Singularity images based on NVIDIA's NGC TensorFlow container (TensorFlow 2.17 with CUDA 12.8), which need only an NVIDIA driver ≥ 570 for GPU runs. A short finetuning run on the target genome before annotating gives the best results, and models download at roughly 100 MB on first use.", "body_md": "**Vipsania is an unsupervised deep learning *ab-initio* gene finder.**\n\nUnlike other deep learning gene finders, Vipsania is never shown a reference annotation. It is trained purely on raw genomic sequences with a masked language modelling objective: nucleotides are hidden and the model learns to predict them from their context. Gene structure emerges as the representation a hidden Markov model inside the network needs in order to explain the sequence, so no curated gene set, no related-species annotation and no protein evidence are required at any point.\n\nFor you, this means:\n\n- **Annotate any eukaryotic genome from a FASTA file alone.** No extrinsic evidence, no reference\nannotation, no gene-structure training data of any kind.\n- **Adapt the model to your species on the fly.** Because training needs nothing but sequence, the\ngenome you want to annotate is itself valid training data. A short finetuning run on your target\ngenome directly before annotating gives the best results — see[Finetuning](#finetuning-recommended) .\n- **Get standard output.** Annotations are written as GenePred, GFF3 or GTF.\n\nVipsania requires `python>=3.12`. It is recommended to install into a clean virtual environment\n(e.g. using `conda`, `pyenv` or `uv`).\n\n``` bash\n$ python -m pip install vipsania\n```\n\nThis installs the `vipsania` command, which is everything you need to annotate a genome. For\ntraining, use the [config files](https://github.com/Gaius-Augustus/Vipsania/blob/main/configs) provided and follow the instructions [here](#training).\n\nAlternatively, you can clone the repository, install Vipsania from source and use the\n[scripts](https://github.com/Gaius-Augustus/Vipsania/blob/main/scripts) for annotation and training.\n\n``` bash\n$ git clone https://github.com/gaius-augustus/vipsania\n$ python -m pip install -e vipsania/\n```\n\nInstalling Vipsania pulls in [bricks2marble](https://github.com/gaius-augustus/bricks2marble) and\n[hidten](https://github.com/gaius-augustus/hidten), which in turn install TensorFlow with CUDA\nsupport. A GPU is strongly recommended; annotating a large genome on CPU only is possible, but\nslow.\n\nIf you prefer not to install Python dependencies locally — or if you are working on an HPC cluster — use the pre-built image from Docker Hub:\n\n**Docker** (requires `sudo` unless your user is in the `docker` group):\n\n``` bash\n$ sudo docker pull gaiusaugustus/vipsania:latest\n$ sudo docker run --rm --gpus all \\\n      -v /path/to/data:/data \\\n      gaiusaugustus/vipsania \\\n      vipsania annotate Fungi genome.fa -o annotation.gff3 --finetune\n```\n\n**Singularity / Apptainer** (no root needed, suited for HPC):\n\n``` bash\n$ singularity pull vipsania.sif docker://gaiusaugustus/vipsania:latest\n$ singularity exec --nv \\\n      -B /path/to/data:/data \\\n      vipsania.sif \\\n      vipsania annotate Fungi /data/genome.fa -o /data/annotation.gff3 --finetune\n```\n\nDrop `--gpus all` (Docker) or `--nv` (Singularity) for CPU-only runs. The\nimage is based on NVIDIA's NGC TensorFlow container (TensorFlow 2.17 with\nCUDA 12.8), so **no CUDA installation on the host is required** — only an\nNVIDIA driver ≥ 570 for GPU runs. The image is `linux/amd64` only and also\nsupports Blackwell GPUs (e.g. RTX PRO 6000, B200, RTX 50xx). Full details, including supported systems, model-cache\npersistence and training, are in [docs/container.md](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/container.md).\n\nAnnotation is done with `vipsania annotate`. It needs a trained model and a FASTA file of the\ngenome you want to annotate.\n\n``` bash\n$ vipsania annotate <model_id> genome.fa -o annotation.gff3 --finetune\n```\n\nThe same command is available as `python -m vipsania annotate ...`, and from a cloned repository as\n`python scripts/annotate.py ...`.\n\nThe first argument names the model to use, either by the **clade** it was trained for or by its\n**model ID**, both listed below. The first time a model is used, Vipsania downloads it (about 100\nMB) and keeps it, so every later annotation with it starts immediately.\n\n``` bash\n$ vipsania annotate Insecta genome.fa -o annotation.gff3 --finetune\n```\n\nA clade name gives you the model we recommend for that clade at the moment. Should a clade get a\nbetter model later, `vipsania download <clade>` picks it up; annotating on its own keeps using the\nmodel it already has. A model ID always refers to that one model.\n\nA model you trained yourself is given in the same way: its ID is the name of its run folder, which\nVipsania looks for in the directories around the working directory. Set `--model_dir` to use a\ndifferent directory that holds the model folder directly.\n\nProgress is printed to the terminal while the genome is annotated, together with a log file next to the output.\n\nVipsania is trained per clade. Use the model of the clade your species belongs to, choosing the most specific one available.\n\n| Clade | Model ID | Training species | Test species | Average locus F1 | Average locus F1 (pretrained) | \n|---|---|---|---|---|---|\n| Alveolata | sd2zcj7u | 30 | 6 | 0.613 | 0.582 | \n| Amoebozoa | ezhpj2qm | 15 | 7 | 0.437 | 0.416 | \n| Arthropoda | ihe1jk30 | 200 | 6 | 0.207 | 0.217 | \n| Chlorophyta | faeijtmk | 26 | 6 | 0.579 | 0.558 | \n| Cnidaria | b5vtieo0 | 82 | 10 | 0.453 | 0.406 | \n| Discoba | gcra9d9y | 20 | 4 | 0.696 | 0.696 | \n| Echinodermata | sx9zjl7p | 52 | 8 | 0.444 | 0.440 | \n| Fungi | fh1kg88z | 200 | 7 | 0.660 | 0.645 | \n| Insecta | 58hsuobw | 200 | 9 | 0.537 | 0.474 | \n| Nematoda | zspca2rb | 73 | 8 | 0.539 | 0.517 | \n| Porifera | 7bjiexcu | 75 | 6 | 0.480 | 0.416 | \n| Rhodophyta | 6kmw3wme | 20 | 5 | 0.405 | 0.301 | \n| Spiralia | v5ej8oyt | 200 | 6 | 0.374 | 0.262 | \n| Stramenopiles | r6p9z9jw | 32 | 6 | 0.426 | 0.413 | \n| Streptophyta | j9m0cdmk | 200 | 13 | 0.602 | 0.579 | \n| Tunicata | hcehc7ff | 34 | 6 | 0.523 | 0.494 | \n| Vertebrata | etb1go6q | 200 | 13 | 0.406 | 0.327 | \n| *none of the above* | cg6grhms | 33 | 7 | 0.436 | 0.416 | \n\nUse the last entry for eukaryotes that belong to none of the clades above — it was trained on\nexactly such species. It has no clade name of its own; pass `Other` or its model ID.\n\nThe two F1 columns are the average locus F1 over the test species of that clade, measured against\ntheir reference annotations: the first with `--finetune`, the second with the pretrained model\nalone. Finetuning helps in almost every clade, which is why we recommend it.\n\nWhichever model you pick, finetuning it on your genome adapts it to your species — see\n[Finetuning](#finetuning-recommended).\n\nThe species every model was trained on are listed in\n[docs/training_species.tsv](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/training_species.tsv), one row per species with its NCBI\nassembly accession. Since Vipsania never learns from annotations, the quality of a species'\nreference annotation, or whether it has one at all, leaves no trace in the model, and unlike for a\nsupervised gene finder, finding your species or a close relative in the list is not an argument\nagainst that model.\n\nThe test species behind the two F1 columns are in\n[docs/test_species.tsv](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/test_species.tsv), with precision, sensitivity and F1 at base,\nexon and locus level for each of them, once with and once without finetuning.\n\nThe first time a model ID is used it is downloaded into `~/.cache/vipsania/models` and read from\nthere ever after. See [docs/download.md](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/download.md) for changing that location, and for\nfetching models in advance on machines that have no internet access when they annotate.\n\nThe output format is chosen by the file suffix of `-o/--output`:\n\n| suffix | format | \n|---|---|\n| `.gp` | GenePred | \n| `.gff3` /`.gff` | GFF3 | \n| `.gtf` | GTF | \n\nIf `-o` is omitted, the annotation is written next to the input FASTA as `vipsania_[model_id].gff`,\nnamed after the model that produced it even when you asked for it by clade. The protein and coding\nsequences of the predicted genes can be written out at the same time with `--protein proteins.fa`\nand `--coding coding.fa`.\n\n``` bash\n$ vipsania annotate <model_id> genome.fa -o annotation.gff3 --finetune\n```\n\n`--finetune` trains the model on the genome you are about to annotate before predicting anything.\nSince Vipsania is unsupervised, this needs nothing except the FASTA file that you already have. The\nmodel adapts to the codon usage, repeat landscape and intron statistics of your species, which\nconsistently improves the annotation.\n\nThe finetuned checkpoint is saved next to the output file, so you can reuse it for further\nannotations of the same genome or closely related species without finetuning again. Use the\n`--model_dir` argument in these cases.\n\nAll options of the annotation are listed in [docs/annotate.md](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/annotate.md), or with\n`vipsania annotate --help`.\n\n[Paludamentum](https://github.com/Gaius-Augustus/Paludamentum) is the Nextflow pipeline around the\ngene finders of the Gaius-Augustus family. With Vipsania as gene finder it splits the genome, runs\n`vipsania annotate` on several GPUs in parallel, and, if you give it proteins, RNA-Seq or Iso-Seq,\nderives high-confidence genes from that evidence and merges them with the Vipsania prediction.\nVipsania is a git submodule of Paludamentum; the `vipsania` command itself does not run the\npipeline.\n\n``` bash\n$ git clone --recursive https://github.com/Gaius-Augustus/Paludamentum\n$ cd Paludamentum && pip install .\n$ paludamentum --genefinder vipsania --nf_config slurm_generic --genome genome.fa --model Fungi \\\n      --proteins proteins.faa\n```\n\nFinetuning is off in the pipeline by default; `--finetune` switches it on and then runs one\nVipsania task on the whole genome. See [docs/pipeline.md](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/pipeline.md).\n\nVipsania models are trained with `vipsania train` on a set of local FASTA files. Ready-to-use\nconfigurations for the three model sizes are in [configs](https://github.com/Gaius-Augustus/Vipsania/blob/main/configs); rather than editing one, you\nlist the FASTA files you want to train on in [configs/train.json](https://github.com/Gaius-Augustus/Vipsania/blob/main/configs/train.json) and pass it\nas an override:\n\n``` bash\n$ vipsania train configs/base_10M.json -oc configs/train.json\n```\n\nTraining runs can be logged to [Weights & Biases](https://wandb.ai) with `--online entity/project`.\nEverything a run produces — configuration, checkpoints and metrics — is written to a folder below\n`./checkpoints`, named after the run. That name is exactly the model ID that `vipsania annotate`\nexpects.\n\nA published model can also be trained further on species of your choice, which extends the\n`--finetune` option of `vipsania annotate` to a set of genomes. See\n[docs/training.md](https://github.com/Gaius-Augustus/Vipsania/blob/main/docs/training.md) for that, for the configuration format, and for what to\nchange when a run does not fit into GPU memory.\n\nVipsania is built on two libraries developed in our group:\n\n- [bricks2marble](https://github.com/gaius-augustus/bricks2marble) — efficient handling of\nnucleotide sequences and genome annotations, plus the pre- and postprocessing around genome\nannotation.\n- [hidten](https://github.com/gaius-augustus/hidten-docs) — hidden Markov models as differentiable,\nhighly parallel layers inside deep learning models.", "url": "https://wpnews.pro/news/annotating-genomes-via-unsupervised-deep-learning", "canonical_source": "https://github.com/gaius-augustus/vipsania", "published_at": "2026-10-16 05:35:46+00:00", "updated_at": "2026-10-16 06:06:44.218338+00:00", "lang": "en", "topics": ["machine-learning", "ai-research", "developer-tools"], "entities": ["Vipsania", "Gaius-Augustus", "TensorFlow", "NVIDIA", "Docker", "Singularity", "bricks2marble", "hidten"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/annotating-genomes-via-unsupervised-deep-learning", "markdown": "https://wpnews.pro/news/annotating-genomes-via-unsupervised-deep-learning.md", "text": "https://wpnews.pro/news/annotating-genomes-via-unsupervised-deep-learning.txt", "jsonld": "https://wpnews.pro/news/annotating-genomes-via-unsupervised-deep-learning.jsonld"}}