{"slug": "what-counts-as-a-large-cosine-similarity", "title": "What counts as a large cosine similarity?", "summary": "In 200-dimensional word embeddings such as gensim's glove-twitter-200, a cosine similarity of 0.656 — corresponding to an angle of about 49° — counts as a large similarity, according to an analysis by John D. Cook. The vector arithmetic \"king\" − \"man\" + \"woman\" ≈ \"queen\" yields a cosine similarity of 0.656 between the computed vector and \"queen\", while the angle between \"king\" and \"fireplace\" is 89.25°. Because nearly all vectors in high dimensions are nearly perpendicular, the proportion of a 200-dimensional sphere's area within 49° of a given point is on the order of 10^−26, versus 17% for a 3-dimensional sphere.", "body_md": "Machine learning represents words as vectors and measures the similarity of words by the angles between the vectors.\n\nFor vectors **x** and **y**,\n\nwhere θ is the angle between the vectors, and so\n\nThis is the cosine similarity between the words represented by **x** and **y**.\n\nSmall angles have large cosines, and so words with larger cosine similarities are closer together than words with smaller cosine similarities. The cosine similarity between a word and itself equals 1, and we’d expect unrelated words to have a cosine similarity near 0.\n\nYou can do a sort of arithmetic with vector embeddings of words. The canonical example is that\n\n“king” − “man” + “woman” ≈ “queen”\n\nImplicit in this equation is that we’re really adding vector representations of the words. Let **a**, **b**, **c**, and **d** be the vector embeddings of the words *king*, *man*, *woman*, and *queen*. What we’re really asserting is that\n\n**a** − **b** + **c** ≈ **d**,\n\nexcept that’s not true! Or at least it’s not true unless you view it in the right context.\n\nThe angle between **a** − **b** + **c** and **d** is about 49°, which corresponds to a cosine similarity of 0.656. Here I’m using the gensim glove-twitter-200 embedding that represents words as 200-dimensional vectors.\n\nThe way to interpret the equation above is not that a 49° degree angle is approximately 0, or that a similarity of 0.656 is approximately 1.\n\nIn high dimensions, such as 200-dimensional word embeddings, nearly all vectors are nearly perpendicular. I wrote a post about this [here](https://www.johndcook.com/blog/2023/08/09/random-points-hypersphere-orthant/). So the angle between randomly selected words will usually be close to 90°, and so in that context an angle of 49° is relatively small. For example, the angle between the vector representations of *king* and *fireplace* is 89.25°.\n\nIf you divide word vectors by their norm, you can think of each vector as a point on a high-dimensional sphere, in our case a sphere in 200 dimensions. The proportion of vectors within 49° of a given point is surprisingly small in high dimensions.\n\nLet’s say our point of interest is the north pole of an *n*-dimensional sphere. We’d like to calculate the proportion of the area of the sphere that is within an angle θ of the pole. I go through the calculations [here](https://www.johndcook.com/blog/2023/08/09/hypersphere-cap/).\n\nWhen *n* = 3, 17% of the area is with 49 degrees of the pole. But when *n* = 200, the proportion is on the order of 10<sup>−26</sup>, essentially zero.\n\nThe vector **d** above representing *queen* is within a relatively tiny region around the vector **a** − **b** + **c**.\n\nIn terms of cosine similarity, 0.656 is a large similarity. Words with a cosine similarity in this range are quite close, even though we wouldn’t normally think of 0.656 being close to 1. In this context, 0.656 *is* close to 1.", "url": "https://wpnews.pro/news/what-counts-as-a-large-cosine-similarity", "canonical_source": "https://www.johndcook.com/blog/2026/09/15/cosine-similarity/", "published_at": "2026-09-15 16:06:02+00:00", "updated_at": "2026-09-15 16:21:45.811734+00:00", "lang": "en", "topics": ["natural-language-processing", "machine-learning", "artificial-intelligence"], "entities": ["John D. Cook", "gensim", "glove-twitter-200"], "alternates": {"html": "https://wpnews.pro/news/what-counts-as-a-large-cosine-similarity", "markdown": "https://wpnews.pro/news/what-counts-as-a-large-cosine-similarity.md", "text": "https://wpnews.pro/news/what-counts-as-a-large-cosine-similarity.txt", "jsonld": "https://wpnews.pro/news/what-counts-as-a-large-cosine-similarity.jsonld"}}