By Evan Vega
Semantic caching, the practice of returning a stored AI answer when a new question means the same thing as an earlier
one, is being sold as a way to cut large-language-model bills. Redis’s product page for its LangCache service says “Save
90% on API costs.” Novel Cognition checked that claim against Redis’s own documentation and found the service is still
in preview, fifteen months after its public preview began, with no published price. Redis’s own savings formula
multiplies the cache hit rate by output-token cost, which puts real savings below the hit rate.
Novel Cognition also measured the trade-off that decides whether a semantic cache helps or hurts: the similarity
threshold. The full analysis, with sources and test data, is at
semanticcache.novcog.us.com.
Redis says its LangCache semantic cache can “Save 90% on API costs,” but the service is still in preview with no published price, and Redis’s own formula puts savings below the hit rate. Novel Cogniti
The documented failure modes are false hits, where a different question gets a stored answer; multi-turn and agent
traffic, where LiteLLM’s documentation warns that consecutive turns look about 0.99 similar and a cache “replays a stale
response”; unscoped caches that can serve one user’s answer to another; and stale answers after policies change. A
Stanford-led NeurIPS 2025 paper found semantic caching “suffers from a high rate of false-positive cache hits” on agent
workloads. Lab research has shown timing attacks that inferred other users’ questions 95.4% of the time after five probes
(IEEE TIFS), and crafted queries that pulled planted answers 80.6% and 88.3% of the time on AWS and Azure semantic-cache
setups (ICML 2026). None reports a production incident.
Provider prompt caching from Anthropic and OpenAI is a different tool: it reuses an identical prompt prefix and the
model still writes a fresh answer. For self-hosting, Novel Cognition’s comparison of thirteen options lists four
ingredients for a homemade cache on Postgres with pgvector or on Redis Open Source: an embedding lookup, a threshold, an
expiry and a partition key per tenant. See where it
breaks and build or buy.
A semantic cache turns each question into an embedding, finds the closest stored question, and returns the stored
answer when the similarity score clears a threshold. Novel Cognition ran 243 hand-labelled customer-support question
pairs through Qwen3-Embedding-0.6B, a small general-purpose embedding model, on its own hardware. “How do I cancel my
subscription?” and “How do I pause my subscription?”, which need different answers, scored 0.8675. “How do I cancel my
subscription?” and “I want to stop being charged every month and end my plan. How?”, which need the same answer, scored
0.6794.
At 0.85, the default threshold Redis documents for LangCache, that model caught none of 101 genuine rewordings and
wrongly matched 29 of 102 look-alike questions (28.4%). Novel Cognition stresses the limits: the pairs were written to
probe the hard cases, and the model is not the one LangCache uses. The finding is that
a threshold only means something for one embedding model on
one kind of traffic.
Published work points the same way. An AWS benchmark published 26 November 2025 found that about 8% of cached
answers missed the user’s intent at every threshold from 0.75 to 0.99, on a dataset of Chatbot Arena prompts plus
generated paraphrases. The vCache paper (arXiv 2502.03771, ICLR 2026) found correct and incorrect cache hits averaged
0.84 and 0.85 similarity. The SCALM study found only 7.5% of questions in 300,000 public chatbot conversations could
have been answered from an earlier one.
LangCache is a semantic caching service that Redis describes as “available as a REST API that stores LLM responses
for fast and cheaper retrieval, built on the Redis vector database.” Applications search it before calling a model and
store the answer after a miss. Its documentation, last updated 5 October 2026, says it is “currently in preview.” The
public preview on Redis Cloud began in July 2025; a self-managed Kubernetes version is in private preview. Since May
2026 it has been sold as part of Redis Iris.
Redis’s launch materials in September 2025 promised savings of “up to 70%.” The current product page says “Save 90%
on API costs.” The showcase customer, Mangoes.ai, a behavioral-health voice assistant, is quoted by Redis as reporting a
70% cache hit rate that “saves 70% of our LLM spend.” That figure is self-reported, with no method given. Redis’s
pricing calculator labels its figures “illustrative,” with final pricing “TBD.” The details are at
semanticcache.novcog.us.com/langcache.
Semantic caching pays on high-repetition traffic with stable answers, such as support bots, FAQ assistants and
internal policy questions, and pays least on open-ended chat, agents and generated content. The deciding work is
measuring how often real traffic repeats and testing the threshold against look-alike questions from that traffic.
Novel Cognition offers a free traffic review at novcog.com/contact; every
source behind this analysis is listed at sources.
This analysis was produced under the Hidden State Drift method, which
separates vendor claims from independent measurement. The ten pre-publication checks behind it are published at
hiddenstatedrift.com/verify. Novel Cognition is not affiliated with
Redis or any vendor discussed.
Part of the Frontier Watch Series: Read the previous investigation
More Coverage:
→ Read this investigation on North Denver Tribune
→ Coverage from Daily Colorado News
