emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity

Researchers have released emb-diversity, a tool for measuring data diversity using embeddings, addressing the lack of standardized tools for embedding-based diversity quantification. The tool works with any embedding model and data type, enabling measurement of stylistic, semantic, language, and speaker diversity in NLP datasets.

arXiv:2607.19848v1 Announce Type: new Abstract: There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmented: While there exist a number of tools for measuring the lexical diversity of texts, researchers lack standardized tools for quantifying diversity based on embeddings. Embedding-based diversity measures are highly flexible: They work with any embedding model and any data that can be embedded, and are thus applicable to many notions of diversity. With emb-diversity, we provide a comprehensive embedding-based diversity measurement tool, spanning a broad range of measures. We demonstrate its potential for several use cases: measuring the stylistic, semantic, language and speaker diversity of datasets. https://github.com/nlpsoc/emb-diversity/