How Can AI Find My Model? A Model-Finding Experimental Study Considering Data Formats, Embeddings, and Retrieval Strategies

A new study from arXiv investigates how AI can retrieve simulation models using natural language queries, finding that data representation, open-source embedding models, and reranking strategies significantly impact retrieval performance. The research provides a baseline for AI-driven model discovery, aiming to improve composability and interoperability in modeling and simulation.

arXiv:2606.30846v1 Announce Type: new Abstract: Discovering simulation models for reuse remains a fundamental challenge in Modeling and Simulation M&S . When many models coexist, identifying those that align with a given modeling intent remains difficult. Recent advances in Artificial Intelligence AI , particularly retrieval-based approaches, offer a promising pathway to operate at this semantic layer. In this paper, we present an experimental study investigating the impact of data representation, transformer-based embedding models, and retrieval strategies on the discovery of simulation models using natural language queries. We evaluated performance across multiple query types using standard information retrieval metrics, including recall@5 and nDCG@5. Results show that data representation matters, open-source embedding models can achieve high performance, and reranking methods are important, especially as query complexity increases. This work provides a baseline for AI-driven model discovery and discusses its role in advancing toward AI-driven composability and interoperability.