{"slug": "vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients", "title": "Vector search: HotKV vs Redis 8 vector sets, from 1 to 100 clients", "summary": "The HotKV team benchmarked its unreleased vector search engine against Redis 8.10.2's vector sets on 999,000 OpenAI text-embedding-ada-002 vectors, finding HotKV served 1.4x Redis's queries per second with one client and 7.9x with 100 clients (51,146 vs 6,510) at a 95% recall target, with lower median latency at every client count. At 99% recall the throughput ratios were 1.8x and 5.4x, and HotKV loaded the vectors in 188 seconds versus 935 seconds for Redis. The team cautioned the results cover one dataset on one machine, an unreleased development build, and did not test Redis's Query Engine or 8-bit mode.", "body_md": "Vector search sits underneath many AI features: retrieval for RAG, semantic caching, agent memory. Redis 8 ships vector sets, a beta data type, built into the server, so it is the obvious comparison. We loaded the same OpenAI embeddings into HotKV and into Redis 8.10.2, ran the same 1,000 queries against each with 1 to 100 concurrent client connections, and measured how accurate the answers were, how many queries per second each engine served, and how long each query took.\n\n**In short.** On 999,000 vectors, at a minimum recall of 95%, HotKV served 1.4 times the queries per second of Redis with one client and 7.9 times with 100 clients (51,146 against 6,510). At 99% recall the ratios were 1.8 and 5.4. HotKV's median latency was lower at every client count. Redis's throughput levelled off by 32 clients, while HotKV's kept growing to 100. HotKV loaded the vectors in 188 s against 935 s for Redis, which was pinned to four vCPUs for its load, and its recall was higher at every search depth. This is one dataset on one machine, the HotKV build is an unreleased development build, and we did not measure Redis's Query Engine or its 8-bit mode, so read it as a measurement, not a ranking.\n\nThe first 999,000 rows of [dbpedia-entities-openai-1M](https://huggingface.co/datasets/KShivendu/dbpedia-entities-openai-1M) by Kumar Shivendu and Nirant Kasliwal (MIT license), and, for a smaller index, the first 100,000 rows. The rows are DBpedia entities embedded with OpenAI `text-embedding-ada-002`, 1,536 dimensions. The last 1,000 rows are the queries, so no row appears in both sets. The answer key is the exact top 10 nearest neighbours of every query, computed by brute force in 64-bit floating point independently of either engine, so both are graded against the same ground truth.\n\n`c8dafd37` (`VSIM.CREATEINDEX`, `VSIM.SET`, `VSIM.SEARCH`), an unreleased development build compiled with default release flags, not `-march=native`. Redis 8.10.2 built from its release tag with a stock `make` (`VADD`, `VSIM`). Both build `M=16` and `ef_construction=200` on both, set explicitly. They are also each engine's own defaults.`NOQUANT`.\nFor search depths (`ef`) of 40, 80, 150 and 300, and 500 on the larger index, we ran the 1,000 queries repeatedly with 1, 2, 4, 8, 16, 32, 50 and 100 clients: one second of warm-up, then four seconds counted, three times each, with the order of the two engines alternating. We report the median. Recall was measured separately, over one connection. HotKV was loaded over 32 connections in every run. The Redis indexes were built once, over 32 connections with `VADD ... CAS`, saved, and loaded for the measurements. The scripts and every trial are in the [benchmark repository](https://github.com/HotKV-LTD/hotkv/tree/main/vector-search).\n\nThe two engines do not reach the same recall at the same `ef`, so we compare throughput at shared minimum recall targets: for each target, the fastest measured search depth that reaches it. The depths and recalls reached are in the tables, because they differ a little. This uses measured depths, not interpolation. Queries per second on 999,000 vectors:\n\n| Recall@10 target | Clients | HotKV | Redis 8.10.2 | HotKV vs Redis | Median latency | \n|---|---|---|---|---|---|\n| 95% or better | 1 | 1,328 (ef 80, 0.970) | 946 (ef 80, 0.962) | 1.40x | 0.77 / 1.07 ms | \n| 95% or better | 8 | 10,342 | 5,531 | 1.87x | 0.79 / 1.43 ms | \n| 95% or better | 16 | 18,500 | 7,265 | 2.55x | 0.87 / 2.15 ms | \n| 95% or better | 50 | 37,508 | 6,825 | 5.50x | 1.33 / 7.17 ms | \n| 95% or better | 100 | 51,146 | 6,510 | 7.86x | 1.91 / 13.49 ms | \n| 99% or better | 1 | 507 (ef 300, 0.993) | 287 (ef 500, 0.994) | 1.77x | 1.99 / 3.31 ms | \n| 99% or better | 16 | 6,961 | 2,322 | 3.00x | 2.30 / 6.70 ms | \n| 99% or better | 50 | 12,393 | 2,483 | 4.99x | 4.02 / 18.91 ms | \n| 99% or better | 100 | 13,843 | 2,583 | 5.36x | 7.11 / 37.65 ms | \n\nMedian latency is HotKV / Redis. The same pattern held for a 90% target: 90,003 against 7,973 queries per second at 100 clients (11.3 times), both at `ef` 40.\n\n| ef | HotKV recall | Redis recall | HotKV, 1 client | Redis, 1 client | HotKV, 50 clients | Redis, 50 clients | HotKV, 100 clients | Redis, 100 clients | \n|---|---|---|---|---|---|---|---|---|\n| 40 | 0.940 | 0.923 | 2,108 | 1,259 | 63,897 | 8,061 | 90,003 | 7,973 | \n| 80 | 0.970 | 0.962 | 1,328 | 946 | 37,508 | 6,825 | 51,146 | 6,510 | \n| 150 | 0.987 | 0.980 | 875 | 636 | 21,912 | 5,241 | 26,122 | 4,942 | \n| 300 | 0.993 | 0.990 | 507 | 396 | 12,393 | 3,454 | 13,843 | 3,402 | \n| 500 | 0.997 | 0.994 | 338 | 287 | 7,688 | 2,483 | 8,843 | 2,583 | \n\nQueries per second on 999,000 vectors. On 100,000 vectors the pattern is the same. At 95% recall HotKV served 1.5 times Redis's queries per second with one client (1,816 against 1,214) and 8.9 times with 100 (63,247 against 7,119).\n\nEach cell is the median of three trials. Half of the cells had a spread between their lowest and highest trial of 1.3% of the median or less, nine in ten 7% or less, and the widest was 26% (Redis, 100,000 vectors, `ef` 300, 50 clients). The report lists every trial.\n\nRecall was measured once per index build. Three independent builds of each engine on 100,000 vectors differed by up to 0.013 at the same depth, so HotKV's higher recall is clear up to `ef` 300 and within build-to-build variation at `ef` 500 on the smaller index.\n\nLoaded over 32 connections each, HotKV took 188 seconds for the 999,000 vectors (5,325 per second) and Redis 935 seconds (1,069 per second), including the time the load generator spent formatting each vector. On 100,000 vectors it was 15 seconds against 57. Redis loaded faster with `CAS` than without in a check on 20,000 vectors (2,279 against 1,488 vectors per second), so we used it. The Redis load ran in an earlier step with Redis pinned to four vCPUs while other checks ran on the machine; HotKV loaded alone on 128 vCPUs. Treat the Redis loading time as a lower bound on its speed, not a like-for-like comparison.\n\nAfter loading 999,000 vectors the HotKV process held 6.77 GB and the Redis process 7.01 GB.\n\nRedis vector sets run each search on its own thread, but the HNSW code has a fixed number of thread slots (32, set when Redis is compiled), and vectors are added on the main thread. In our runs, from 32 clients up, Redis kept between 6 and 30 of the machine's vCPUs busy and its throughput had levelled off. HotKV kept about 92 vCPUs busy at 100 clients on `ef` 80 and its throughput was still growing. That is how each engine behaves on this machine, not something the benchmark configured.\n\nThese are HotKV Ltd's measurements of the versions and configuration above, not a claim about every Redis deployment.\n\n`FT.CREATE` and `FT.SEARCH`. We tested vector sets only, so the results say nothing about the Query Engine.\nVector search is built into the same server as HotKV's prompt cache, and the same vector index backs its semantic cache and agent memory. Vector search is included on every HotKV Cloud plan. The prompt cache, semantic cache and agent memory come with the Performance plan and above. [Start on HotKV Cloud](https://console.hotkv.com/register), or [talk to sales](mailto:sales@hotkv.com?subject=HotKV%20vector%20search) to benchmark it against your own embeddings.", "url": "https://wpnews.pro/news/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients", "canonical_source": "https://dev.to/hotkv/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients-2d4o", "published_at": "2026-11-05 01:39:46+00:00", "updated_at": "2026-11-05 01:52:10.010874+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "machine-learning"], "entities": ["HotKV", "Redis", "OpenAI", "Kumar Shivendu", "Nirant Kasilwal", "DBpedia"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients", "markdown": "https://wpnews.pro/news/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients.md", "text": "https://wpnews.pro/news/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients.txt", "jsonld": "https://wpnews.pro/news/vector-search-hotkv-vs-redis-8-vector-sets-from-1-to-100-clients.jsonld"}}