Fast And Accurate Text Content File Type Identification

A new neural network model for identifying text content file types, especially source code, is approximately four times faster than Google's Magika and 28% smaller in size, according to an arXiv paper (2609.21306v1). The authors report their model is also more accurate on average than existing tools for text-content file-type identification, tested on open-source files. The work targets cybersecurity use cases where magic numbers and file extensions cannot be trusted.

arXiv:2609.21306v1 Announce Type: new Abstract: A common requirement across organizations is to have a tool that can identify file types based on their contents, particularly in the cybersecurity domain where magic numbers and file extensions can not be trusted. While existing tools work well in practice, there is plenty of room for improvement either in terms of computational load and time for detection in the case of model based tools like Magika or in terms of accuracy of detection in the case of file parsing tools that use programming language constructs. In this study, we propose a neural network model for identification of types of text content files, especially source code, that is more accurate and faster than other available tools. Our experiments on open-source files indicate that it is not only more accurate on average for text-content file-type identification, but also approximately four times faster than Magika, while being 28% smaller in size.