On the Diversity of Analogy Making in Large Language Models

A new study from arXiv (2608.03233v1) evaluating ten state-of-the-art open- and closed-source large language models finds that LLMs exhibit a concerning tendency toward domain homogeneity in analogy making, often drawing from a narrow set of target domains, which limits diversity across queries and within models. The research also reveals a fundamental trade-off in existing diversity-enhancement methods, where increasing output diversity often reduces output quality, and causal analysis of LLM information flow suggests differences in model-sensitive regions may explain this trade-off.

arXiv:2608.03233v1 Announce Type: new Abstract: Large Language Models LLMs have demonstrated remarkable potential for analogy making, a core cognitive capability that drives novelty and creativity. While prior research has extensively investigated the applications and underlying mechanisms of LLM-based analogy making, its output diversity remains largely unexplored, despite being essential for broadening cross-domain connections and fostering scientific innovation. In this work, we present a comprehensive evaluation of analogy diversity across ten state-of-the-art open- and closed-source LLMs. Our findings highlight a concerning issue of domain homogeneity, a prevalent tendency for LLMs to generate analogies from a narrow set of target domains, limiting both inter-query and intra-model diversity. Furthermore, our analysis reveals a fundamental trade-off in existing LLM diversity-enhancement methods: increasing output diversity often comes at the expense of output quality. Finally, our causal analysis of LLM information flow reveals substantial differences in the model-sensitive regions governing analogy diversity across LLMs, suggesting a potential mechanism for the observed diversity-quality trade-off. To our knowledge, this is among the first studies to systematically investigate output diversity in LLM-based analogy making.