Andrej Karpathy on X: "Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet."
Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.
Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.
fun detail: a model that just answers "water" every time scores roughly 70% here, since most of the planet is ocean. so Sonnet 4.5 at 60% is actually worse than never guessing land. 90%+ is where it really starts knowing the map
remember this old post of mine? Tried it with Jev---it seems to have a grasp of the globe on par with some of the best models a year ago due to the architecture & low cost, it was also feasible to extract a labeled map of continents and countries. lots of interesting details