Annotating Genomes via Unsupervised Deep Learning

Vipsania is an unsupervised deep learning ab-initio gene finder.

Unlike other deep learning gene finders, Vipsania is never shown a reference annotation. It is trained purely on raw genomic sequences with a masked language modelling objective: nucleotides are hidden and the model learns to predict them from their context. Gene structure emerges as the representation a hidden Markov model inside the network needs in order to explain the sequence, so no curated gene set, no related-species annotation and no protein evidence are required at any point.

For you, this means:

• Annotate any eukaryotic genome from a FASTA file alone. No extrinsic evidence, no reference annotation, no gene-structure training data of any kind.
• Adapt the model to your species on the fly. Because training needs nothing but sequence, the genome you want to annotate is itself valid training data. A short finetuning run on your target genome directly before annotating gives the best results — seeFinetuning .
• Get standard output. Annotations are written as GenePred, GFF3 or GTF.

Vipsania requires python>=3.12. It is recommended to install into a clean virtual environment (e.g. using conda, pyenv or uv).

$ python -m pip install vipsania

This installs the vipsania command, which is everything you need to annotate a genome. For training, use the config files provided and follow the instructions here.

Alternatively, you can clone the repository, install Vipsania from source and use the scripts for annotation and training.

$ git clone https://github.com/gaius-augustus/vipsania
$ python -m pip install -e vipsania/

Installing Vipsania pulls in bricks2marble and hidten, which in turn install TensorFlow with CUDA support. A GPU is strongly recommended; annotating a large genome on CPU only is possible, but slow.

If you prefer not to install Python dependencies locally — or if you are working on an HPC cluster — use the pre-built image from Docker Hub:

Docker (requires sudo unless your user is in the docker group):

$ sudo docker pull gaiusaugustus/vipsania:latest
$ sudo docker run --rm --gpus all \
      -v /path/to/data:/data \
      gaiusaugustus/vipsania \
      vipsania annotate Fungi genome.fa -o annotation.gff3 --finetune

Singularity / Apptainer (no root needed, suited for HPC):

$ singularity pull vipsania.sif docker://gaiusaugustus/vipsania:latest
$ singularity exec --nv \
      -B /path/to/data:/data \
      vipsania.sif \
      vipsania annotate Fungi /data/genome.fa -o /data/annotation.gff3 --finetune

Drop --gpus all (Docker) or --nv (Singularity) for CPU-only runs. The image is based on NVIDIA's NGC TensorFlow container (TensorFlow 2.17 with CUDA 12.8), so no CUDA installation on the host is required — only an NVIDIA driver ≥ 570 for GPU runs. The image is linux/amd64 only and also supports Blackwell GPUs (e.g. RTX PRO 6000, B200, RTX 50xx). Full details, including supported systems, model-cache persistence and training, are in docs/container.md.

Annotation is done with vipsania annotate. It needs a trained model and a FASTA file of the genome you want to annotate.

$ vipsania annotate <model_id> genome.fa -o annotation.gff3 --finetune

The same command is available as python -m vipsania annotate ..., and from a cloned repository as python scripts/annotate.py ....

The first argument names the model to use, either by the clade it was trained for or by its model ID, both listed below. The first time a model is used, Vipsania downloads it (about 100 MB) and keeps it, so every later annotation with it starts immediately.

$ vipsania annotate Insecta genome.fa -o annotation.gff3 --finetune

A clade name gives you the model we recommend for that clade at the moment. Should a clade get a better model later, vipsania download <clade> picks it up; annotating on its own keeps using the model it already has. A model ID always refers to that one model.

A model you trained yourself is given in the same way: its ID is the name of its run folder, which Vipsania looks for in the directories around the working directory. Set --model_dir to use a different directory that holds the model folder directly.

Progress is printed to the terminal while the genome is annotated, together with a log file next to the output.

Vipsania is trained per clade. Use the model of the clade your species belongs to, choosing the most specific one available.

Use the last entry for eukaryotes that belong to none of the clades above — it was trained on exactly such species. It has no clade name of its own; pass Other or its model ID.

The two F1 columns are the average locus F1 over the test species of that clade, measured against their reference annotations: the first with --finetune, the second with the pretrained model alone. Finetuning helps in almost every clade, which is why we recommend it.

Whichever model you pick, finetuning it on your genome adapts it to your species — see Finetuning.

The species every model was trained on are listed in docs/training_species.tsv, one row per species with its NCBI assembly accession. Since Vipsania never learns from annotations, the quality of a species' reference annotation, or whether it has one at all, leaves no trace in the model, and unlike for a supervised gene finder, finding your species or a close relative in the list is not an argument against that model.

The test species behind the two F1 columns are in docs/test_species.tsv, with precision, sensitivity and F1 at base, exon and locus level for each of them, once with and once without finetuning.

The first time a model ID is used it is downloaded into ~/.cache/vipsania/models and read from there ever after. See docs/download.md for changing that location, and for fetching models in advance on machines that have no internet access when they annotate.

The output format is chosen by the file suffix of -o/--output:

If -o is omitted, the annotation is written next to the input FASTA as vipsania_[model_id].gff, named after the model that produced it even when you asked for it by clade. The protein and coding sequences of the predicted genes can be written out at the same time with --protein proteins.fa and --coding coding.fa.

$ vipsania annotate <model_id> genome.fa -o annotation.gff3 --finetune

--finetune trains the model on the genome you are about to annotate before predicting anything. Since Vipsania is unsupervised, this needs nothing except the FASTA file that you already have. The model adapts to the codon usage, repeat landscape and intron statistics of your species, which consistently improves the annotation.

The finetuned checkpoint is saved next to the output file, so you can reuse it for further annotations of the same genome or closely related species without finetuning again. Use the --model_dir argument in these cases.

All options of the annotation are listed in docs/annotate.md, or with vipsania annotate --help.

Paludamentum is the Nextflow pipeline around the gene finders of the Gaius-Augustus family. With Vipsania as gene finder it splits the genome, runs vipsania annotate on several GPUs in parallel, and, if you give it proteins, RNA-Seq or Iso-Seq, derives high-confidence genes from that evidence and merges them with the Vipsania prediction. Vipsania is a git submodule of Paludamentum; the vipsania command itself does not run the pipeline.

$ git clone --recursive https://github.com/Gaius-Augustus/Paludamentum
$ cd Paludamentum && pip install .
$ paludamentum --genefinder vipsania --nf_config slurm_generic --genome genome.fa --model Fungi \
      --proteins proteins.faa

Finetuning is off in the pipeline by default; --finetune switches it on and then runs one Vipsania task on the whole genome. See docs/pipeline.md.

Vipsania models are trained with vipsania train on a set of local FASTA files. Ready-to-use configurations for the three model sizes are in configs; rather than editing one, you list the FASTA files you want to train on in configs/train.json and pass it as an override:

$ vipsania train configs/base_10M.json -oc configs/train.json

Training runs can be logged to Weights & Biases with --online entity/project. Everything a run produces — configuration, checkpoints and metrics — is written to a folder below ./checkpoints, named after the run. That name is exactly the model ID that vipsania annotate expects.

A published model can also be trained further on species of your choice, which extends the --finetune option of vipsania annotate to a set of genomes. See docs/training.md for that, for the configuration format, and for what to change when a run does not fit into GPU memory.

Vipsania is built on two libraries developed in our group:

• bricks2marble — efficient handling of nucleotide sequences and genome annotations, plus the pre- and postprocessing around genome annotation.
• hidten — hidden Markov models as differentiable, highly parallel layers inside deep learning models.