Comparing Semantic Navigation in Humans and Large Language Models using Natural Language Processing

Researchers compared semantic memory retrieval in humans and three large language models (GPT-4o, Gemini-2.5-Pro, Claude-Sonnet-4.5) using verbal fluency data and trajectory-based NLP metrics. Humans exhibited higher entropy, larger semantic steps, and broader dispersion than all LLMs, indicating more variable and exploratory search. No temperature setting in the LLMs reproduced the complete human profile, suggesting current models fail to replicate the distinctive balance between local exploitation and global exploration in human semantic search.

arXiv:2607.12195v1 Announce Type: new Abstract: Semantic memory retrieval can be conceptualized as navigation through conceptual space. We compared semantic search dynamics between humans and three large language models GPT-4o, Gemini-2.5-Pro, Claude-Sonnet-4.5 using verbal fluency data. By applying trajectory-based NLP metrics to the items generated by 82 human participants and LLM output across eight temperature settings, we quantified three complementary dimensions: entropy step size predictability , distance to next successive semantic steps , and distance to centroid global dispersion . Humans exhibited higher entropy, larger semantic steps and broader dispersion than all LLMs, indicating more variable and exploratory search. Temperature tuning produced only partial alignments, as individual metrics matched between humans and LLMs at specific settings, but no configuration reproduced the complete human profile in all dimensions . These findings suggest that human semantic search implements a distinctive balance between local exploitation and global exploration that current model architectures fail to reproduce.