Redis built a cache that cuts LLM costs

Redis's LangCache delivers a 70% cache hit rate that cuts LLM spend by 70% and runs 4X faster for a voice app used in patient care, according to a customer quoted by Redis. The customer said the app handles specific treatment questions and requires absolute accuracy, and credited LangCache with addressing cost concerns for high usage while improving real-time patient interactions.

"Our voice app for patient care gets a lot of specific treatment questions, so it has to be absolutely accurate, and that's what LangCache does. I was worried about LLM costs for high usage, but with LangCache, we're getting a 70% cache hit rate, which saves 70% of our LLM spend. On top of that, it’s 4X faster, which makes a huge difference for real-time patient interactions."