CacheVerifier, check if verifying LLM cache hits helps your traffic CacheVerifier launched a Python client for its hosted API that approves or rejects semantic-cache hits before an application serves a cached answer, aimed at catching similarity matches that are close but wrong. The client requires Python 3.9+ and depends only on httpx at runtime, with opt-in extras for a GPTCache adapter and an offline Health Check; verify() defaults to a 1s timeout and fails closed, returning approved=False with degraded=True on timeouts, connection errors or 5xx responses, while 4xx errors raise CacheVerifierError. Self-serve verification and fine-tuning are free, and the company states there is no formal uptime SLA yet, which it cites as the reason the fallback path is a built-in default. Python client for CacheVerifier https://www.cacheverifier.com — a hosted API that verifies semantic-cache hits. Given a query and a candidate cached answer, it approves or rejects serving that answer from cache, so a similarity match that is close but wrong doesn't become a silent error in your app. CacheVerifier does not run your cache or do similarity search. Your cache backend does its own lookup first; you call verify only on the candidates in the similarity "gray zone", where a plain threshold match might be wrong. - Docs / API reference: https://www.cacheverifier.com/docs https://www.cacheverifier.com/docs - Why similarity ≠ correctness: https://www.cacheverifier.com/why-similarity-fails https://www.cacheverifier.com/why-similarity-fails - The research behind it paper + benchmarks : https://github.com/imxinchengyou/CacheVerifier https://github.com/imxinchengyou/CacheVerifier pip install cacheverifier with the GPTCache adapter: pip install "cacheverifier gptcache " with the offline Health Check adds torch + sentence-transformers : pip install "cacheverifier healthcheck " Requires Python 3.9+. The only runtime dependency is httpx — the extras above are opt-in. Get a free API key at https://www.cacheverifier.com https://www.cacheverifier.com self-serve verify and fine-tuning are free forever, no card . python from cacheverifier import CacheVerifier cv = CacheVerifier api key="cv ..." query = "how do I cancel my subscription" candidate = "Go to Settings Billing Pause subscription for a month." from your cache result = cv.verify query, candidate if result.approved: answer = candidate verified hit — skip the LLM call else: answer = call your llm query not trustworthy — fall through Later, once you know if it was actually right thumbs-down, reopened ticket, ... : cv.feedback query, answer, was correct=True, similarity score=0.86 verify returns a VerifyResult : | field | meaning | |---|---| | approved | serve the cached answer True or fall through False | | score / threshold | approved is score = threshold | | model version | "stock" , "v