Small businesses get the same ten questions all day: opening hours, delivery areas, how to book. Sending each one to a language model is wasted money. A small cache is one of the simplest ways to control AI automation costs for a small business. import hashlib, re, sqlite3, time db = sqlite3.connec
t("answers.db") db.execute("""CREATE TABLE IF NOT EXISTS cache( key TEXT PRIMARY KEY, question TEXT, answer TEXT, created REAL, hits INTEGER DEFAULT 0)""") def normalise(q: str) -> str: q = q.lower() q = re.sub(r"[^a-z0-9 ]+", " ", q) return " ".join(q.split()) def key(q: str) -> str: return hashlib.sha256(normalise(q).encode()).hexdigest() MAX_AGE = 7 * 24 * 3600 # refresh weekly so answers do not go stale def answer(q, call_model): k = key(q) row = db.execute("SELECT answer, created FROM cache WHERE key=?", (k,)).fetchone() if row and time.time() - row[1] < MAX_AGE: db.execute("UPDATE cache SET hits = hits + 1 WHERE key=?", (k,)) db.commit() return row[0], "cache" a = call_model(q) db.execute("REPLACE INTO cache(key, question, answer, created) VALUES (?,?,?,?)", (k, q, a, time.time())) db.commit() return a, "model" Cache only general questions. Never cache answers that include a customer's name, order or account details. Expire entries when prices, hours or policies change (clear the table on any website update). Log cache versus model so you can see what the cache actually saves.
Exact matching catches more than you expect once text is normalised. Fuzzy or embedding matching adds complexity; measure first. Why we often prefer small custom pieces like this over packaged bots: why small custom pieces often beat packaged bots. Disclosure: I co-founded NxFlowAI, an automation agency.
Key Takeaways #
- •Never cache answers that include a customer's name, order or account details.
- •This story was reported by Dev.to , covering developments in thedev space.
- •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.
📖 Continue reading the full article: