We are pleased to announce the release of Annif 1.1!
Annif is a multi-algorithm automated subject indexing tool intended for libraries, archives and museums. It suggest subjects or topics from a predefined vocabulary which can be a thesaurus, ontology or just a list of subjects. The number of the subjects in the vocabulary can be large, tens of thousands or even more, and thus the task Annif performs can be called extreme multilabel classification.
Annif uses more traditional machine learning techniques, not LLMs, which makes it very fast in inference: typically it gives subjects for a text corresponding to a PDF of tens of pages in less than one second. Annif has a CLI for administrative tasks and a REST API for end users. Its development started and continues at the National Library of Finland, but all are welcome to join in!
Regarding Hugging Face, Annif 1.1 introduced annif upload and annif download commands which can be used to push and pull a set of selected projects and vocabularies to and from a Hugging Face Hub repository.
Check out these resources:
PS Maybe someone could forward my above message to the HF posts feed? I’m still in the waitlist for it, so can’t post there myself.
#
This release introduces language detection capabilities in the REST API and CLI, improves Hugging Face Hub integration, and also includes the usual maintenance work and minor bug fixes.
The new REST API endpoint /v1/detect-language expects POST requests that contain a JSON object with the text whose language is to be analyzed and a list of candidate languages. Similarly, the CLI has a new command annif detect-language. Annif projects are typically language specific, so a text of a given language needs to be processed with a project intended for that language; the language detection feature can help in this. For details see this Wiki page. The language detection is performed with the Simplemma library by @adbar et al.
The annif download command has a new --trust-repo option, which needs to be used if the repository to download from has not been used previously (that is if the repository does not appear in the local Hugging Face Hub cache). This option is introduced to raise awareness of the risks of down projects from the internet; the project downloads should only be done from trusted sources. For more information see the Hugging Face Hub documentation.
This release also includes automation of down the NLTK datapackage used for tokenization to simplify Annif installation. Maintenance tasks include upgrading dependencies, including a new version of Simplemma that allows better control over memory usage. The bug fixes include restoring the --host option of the annif run command.
Python 3.12 is now fully supported (previously NN-ensemble and STWFSA backends were not supported on Python 3.12).
#
#
- NN ensemble projects trained with Annif v1.1 or older need to be retrained.
- For other projects, the warnings by SciKit-learn are harmless.
Have you been using Annif for subject indexing or classification? What do you think of it?
We’re interested in your feedback!
The Annif users survey is open until November 30: https://forms.gle/P7jGoPMbEAJnD9zw9 Annif 1.3 has been released!
This release introduces a new EstNLTK analyzer, improves the performance of the MLLM backend and fixes minor bugs.
The key enhancement of this release is the addition of a new analyzer for lemmatization using EstNLTK, which supports the Estonian language. This analyzer needs to be installed separately, see the Optional features and dependencies in Wiki. Note that the indirect dependencies of EstNLTK are quite large, requiring around 500 MB of libraries.
Another improvement is the optimization of the ambiguity feature calculation in the MLLM algorithm. Previously, the calculation could be slow, especially when dealing with a large number of matches when using a large vocabulary such as GND. This optimization addresses the quadratic nature of the ambiguity calculation, and is expected to greatly reduce the processing time of some documents.
This release also includes maintenance updates and bug fixes. The file permissions issue, where Annif did not adhere to the umask setting for data files, has been resolved, thus easing Annif use in multiuser environments.
Supported Python versions:
- 3.9, 3.10, 3.11, and 3.12
Backward compatibility:
- The projects trained with Annif v1.2 remain working.
Last November we organized a survey for Annif users, and now the results have been published in the Doria repository of the National Library of Finland: https://www.doria.fi/bitstream/handle/10024/190930/Annif%20Users%20Survey.pdf
The report includes an overview of:
- The vocabularies and datasets that are used with Annif
- The workflows that Annif is integrated with
- The problems Annif users are facing
The average ratings for various aspects and features of Annif given by users are shown. In short, in a scale from 1 to 5, the ratings are:
- Overall: 4.4
- Features and functions: 4.1
- Documentation: 4.5
- Smoothness of initial setup: 4.2
- Usability: 4.4
- Achieved quality of subject suggestions: 3.6
The survey also gathered user views on the improvements and new features, which are briefly discussed in the report.
We (@osma, @MonaLehtinen & me, i.e. the Annif team at the National Library of Finland) recently took part in the LLMs4Subjects challenge at the SemEval-2025 workshop. The task was to use large language models (LLMs) to generate good quality subject indexing for bibliographic records, i.e. titles and abstracts.
We are glad to report that our system performed well; it was ranked
1st in the category where the full vocabulary was used
2nd in the smaller vocabulary category
4th in the qualitative evaluations.
14 participating teams developed their own solutions for generating subject headings and the output of each system was assessed using both quantitative and qualitative evaluations. Research papers about most of the systems are going to be published around the time of the workshop in late July, and many pre-prints are already available.
We applied Annif together with several LLMs that we used to preprocess the data sets: translated the GND vocabulary terms to English, translated bibliographic records into English and German as required, and generated additional synthetic training data. After the preprocessing, we used the traditional machine learning algorithms in Annif as well as the experimental XTransformer algorithm that is based on language models. We also combined the subject suggestions generated using English and German language records in a novel way.
More information can be found in our system description preprint: Paper page - Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs
See also the task description preprint: Paper page - SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog
The Annif models trained for this task are available here:
NatLibFi/Annif-LLMs4Subjects-data Recently we investigated how the relevance of keyword suggestions produced by Annif has evolved in the publication repositories NatLibFi, see the report (in Finnish). The results provide insights into the development and usefulness of AI-based subject indexing.
The plot shows the the scores averaged over months (dots) and between Annif model updates (horizontal lines): the subject suggestion quality has essentially remained constant since the third model update:
Annif 1.4 has been released!
https://github.com/NatLibFi/Annif/releases/tag/v1.4.0 This release introduces three new corpus formats: a JSON-based full text corpus format (one file per document) and two short-text formats, one based on JSON Lines and another based on CSV. All the new corpus formats include support for document IDs as well as metadata: it is now possible to include structured information such as titles and abstracts for documents. This flexibility is intended to improve the handling of documents that require additional context beyond just the text itself; projects may be configured to operate only on specific metadata fields using the new select transform. All the new corpus formats can be used alongside existing formats.
It is now possible to exclude and include subjects from a vocabulary. Excluding individual concepts can be useful in cases where algorithms frequently produce incorrect subject suggestions. Using exclude and include rules, it is also possible to define more specialized projects that operate on only one type or class of concepts.
Several improvements have been made to the REST API, including exposing vocabulary information via the vocabs method and disabling the learn method by default (controlled by the allow_learn setting in the NN ensemble backend).
The annif index command can now be used on short-text corpus formats (TSV, CSV or JSON Lines) in addition to full text formats (TXT+TSV or JSON). In the case of short-text formats, output including the suggested subjects and their scores is produced in JSON Lines format.
The hyperopt command has been enhanced to better support parallel processing on multiple CPU cores, which can significantly reduce overall processing time.
This release also adds support for Python 3.13, ensuring compatibility with the latest Python version. Furthermore, the tfidf backend has been refactored to eliminate the dependency on gensim, which addresses compatibility issues and simplifies the codebase. Support for Python 3.9 has been dropped. Various maintenance updates and bug fixes are included, such as resolving warnings related to Click and upgrading many libraries to more recent versions.
Special thanks to the German National Library for their work on proposing, specifying and testing the new features in this release!
Supported Python versions:
- 3.10, 3.11, 3.12 and 3.13
Backward compatibility:
tfidf projects trained with Annif 1.3 or older need to be retrained. #
For other projects, the warnings by SciKit-learn are harmless. #
This is very likely the last Annif minor release to support the current fasttext backend, because the original fastText library is no longer maintained and there are compatibility issues with other libraries. We are looking for alternative implementations of fasttext .
Annif 1.5 has been released!
This release includes notable backend enhancements: new EBM and dm_rerank backends are added and the existing NN ensemble backend is reimplemented using PyTorch.
The EBM stands for embedding-based matching. The approach uses a sentence transformer model to vectorize both the labels in the subject vocabulary and input texts to numerical vectors. The vectors of input texts are compared to those of the vocabulary to give candidate subjects, which are then ranked utilizing a model fit to the features of the training documents to give the final suggestions. The backend has been developed by the German National Library (DNB) and it uses the ebm4subjects library.
The dm_rerank is an ensemble style backend that uses a decision model service such as Jev to re-score/re-rank the candidate subjects proposed by the source backends. It supports any decision model with a /v1/systemone API endpoint implemented in recent versions of llama.cpp
Other enhancements include parallelization in the MLLM backend and in the optimization of limit and threshold values, better handling of vocabularies in ensemble projects and adding Pareto-front output to the optimize CLI command. The project configurations can now contain environment variables, which is intended for possibly private information like URLs of external services.
The project’s security posture has been improved by switching to use (PyPI) Trusted Publishing, dependency cooldowns and network-traffic monitoring of CI/CD runs. We have also transferred the PyPI project under NatLibFi organization account and enabled immutability of the GitHub releases.
Maintenance changes include switching from Poetry to uv for dependency management. As always, also some bugs have been fixed.