I'm afraid of spiders. So I made AI look at 2k of them Gemini 3.8 Flash identified spider species correctly on 997 of 2,000 photos, a 49.85% accuracy rate that was the highest among nine AI models tested in a benchmark built from research-grade iNaturalist observations covering 671 species and subspecies. The multiple-choice test, run on the same 2,000-photo subset drawn from 2,183 filtered images, gave each model a list of 20 possible species names per photo including the correct answer. The benchmark was inspired by Piotr Migdał's post on identifying mushrooms with AI. ← All posts https://labqoat.com/blog I’m Afraid of Spiders. So I Made AI Look at 2,000 of Them. In this article I’m afraid of spiders. A lot. 🕷️ My identification system currently consists of “the one with long legs” and “the short but big one.” There’s also “where the fuck did it go,” but that’s more of an emergency than a classification. So I got curious: how well could AI actually identify them? I tested nine models on the same 2,000 spider photos to see how often they got the species right. Looking at this many spiders was not a comfortable experience. This post was heavily inspired by Piotr Migdał’s post on identifying mushrooms with AI https://quesma.com/blog/mushroom-llm-vision/ . First, approximately ten seconds of biology first-approximately-ten-seconds-of-biology A family is a broad group of related spiders. A genus is a smaller group inside it, and a species is the specific kind of spider. For example: Araneidae → Araneus → Araneus diadematus , the European garden spider https://en.wikipedia.org/wiki/Araneus diadematus . In that last name, Araneus is the genus. That’s enough biology for now. How the benchmark works how-the-benchmark-works Each model got the same 2,000 photos , covering 671 species and subspecies , and a list of 20 possible names for each photo, including the correct answer. The task was to identify the spider’s species from the photo and return the matching name from the list. I gave each model the same possible answers so I could compare how well they distinguished the species, with fewer ambiguities in scoring. That makes this a multiple-choice identification test , and the choices themselves can help. I built the dataset from research-grade iNaturalist https://www.inaturalist.org/ observations, using the community’s species identifications as the expected answers. The species list comes from a Polish spider checklist, but the photos were taken worldwide. After filtering out label mismatches and unsuitable images, 2,183 photos remained, and I used the same 2,000-photo subset https://github.com/qforge-dev/spider-bench/blob/main/data/benchmarks/runs/gemini-20260910-230024/tasks.jsonl for every model. Let’s look at a few examples lets-look-at-a-few-examples I’d call any of these “spider,” but… 😅 According to iNaturalist https://www.inaturalist.org/observations/24731930 , it’s a Red-bellied Jumping Spider Philaeus chrysops . What did the models say? 9 of 9 picked the expected species. | Model answers for Philaeus chrysops | | | |---|---|---| | Model | Answer | Result | |---|---|---| | Gemini 3.8 Flash | Philaeus chrysopsCorrect | Correct | | GPT-6 Astra | Philaeus chrysopsCorrect | Correct | | Claude Fable 5.1 | Philaeus chrysopsCorrect | Correct | | Muse Spark 1.3