I Added Retry Logic to My SQLite Failure Library. Here's the Exponential Backoff Pattern That Works.

A developer added retry logic with exponential backoff and jitter to an SQLite failure library to handle 'database is locked' errors under concurrent writes. The solution combines a retry wrapper, connection pooling, and WAL journal mode to reduce write contention in production.

My SQLite failure library started throwing "database is locked" errors under concurrent load. Multiple agents writing failures simultaneously caused write contention. I needed retry logic — but not the dumb kind. The problem: SQLite's default timeout is 5 seconds. Under heavy concurrent writes, agents timeout before the lock is released. The fix is a retry wrapper with exponential backoff and jitter: python import sqlite3 import time import random from typing import Optional DB PATH = Path.home / ".mcp" / "failures.db" def execute with retry func, max retries=5, base delay=0.1 : """Execute a database function with exponential backoff retry.""" for attempt in range max retries : try: return func except sqlite3.OperationalError as e: if "database is locked" not in str e : raise if attempt == max retries - 1: raise Exponential backoff with jitter delay = base delay 2 attempt + random.uniform 0, 0.05 time.sleep delay return func def log failure task: str, error: str, attempted fix: str, result: str, env: str = None : """Log a failure with retry logic.""" def insert : conn = sqlite3.connect str DB PATH , timeout=30 conn.execute "INSERT INTO failures task, error, attempted fix, result, timestamp, env VALUES ?, ?, ?, ?, ?, ? ", task, error, attempted fix, result, time.time , env conn.commit conn.close execute with retry insert, max retries=5, base delay=0.1 The key parameters: Exponential backoff works because it spreads out retry attempts. When multiple agents hit the same lock simultaneously, they don't all retry at the same time. The jitter ensures they retry at slightly different intervals. The timeout parameter is the first line of defense — SQLite itself waits for the lock before raising an error. The retry wrapper handles the case where the timeout expires. I also added a connection pool to reduce the number of simultaneous connections: python class ConnectionPool: def init self, db path, pool size=5 : self.db path = db path self.pool size = pool size self. pool = def get self : if self. pool: return self. pool.pop return sqlite3.connect str self.db path , timeout=30 def put self, conn : if len self. pool < self.pool size: self. pool.append conn else: conn.close pool = ConnectionPool DB PATH def log failure task: str, error: str, attempted fix: str, result: str, env: str = None : conn = pool.get try: conn.execute "INSERT INTO failures task, error, attempted fix, result, timestamp, env VALUES ?, ?, ?, ?, ?, ? ", task, error, attempted fix, result, time.time , env conn.commit finally: pool.put conn conn = sqlite3.connect str DB PATH , timeout=30 conn.execute "PRAGMA journal mode=WAL" Busy timeout vs retry : SQLite's timeout parameter handles the first wait. The retry wrapper handles cases where the timeout expires. Don't rely on just one. Deadlock detection : If you see consistent "database is locked" errors even with retries, you may have a deadlock. Add logging to identify which operations are holding locks: python import traceback def log failure with deadlock detection ... : try: insert except sqlite3.OperationalError as e: if "database is locked" in str e : print f"Deadlock detected: {traceback.format exc }" raise Here's the complete retry + pool + WAL setup I'm running in production: python import sqlite3 import time import random from pathlib import Path DB PATH = Path.home / ".mcp" / "failures.db" def init db : conn = sqlite3.connect str DB PATH , timeout=30 conn.execute "PRAGMA journal mode=WAL" conn.execute "PRAGMA busy timeout=30000" conn.execute """ CREATE TABLE IF NOT EXISTS failures id INTEGER PRIMARY KEY AUTOINCREMENT, task TEXT NOT NULL, error TEXT NOT NULL, attempted fix TEXT, result TEXT NOT NULL, timestamp REAL NOT NULL, env TEXT """ conn.execute "CREATE INDEX IF NOT EXISTS idx task ON failures task " conn.execute "CREATE INDEX IF NOT EXISTS idx timestamp ON failures timestamp " conn.commit conn.close class RetryConfig: def init self, max retries=5, base delay=0.1, max delay=2.0, jitter=True : self.max retries = max retries self.base delay = base delay self.max delay = max delay self.jitter = jitter def execute with retry func, config=None : if config is None: config = RetryConfig last error = None for attempt in range config.max retries : try: return func except sqlite3.OperationalError as e: last error = e if "database is locked" not in str e : raise if attempt == config.max retries - 1: raise delay = min config.base delay 2 attempt , config.max delay if config.jitter: delay += random.uniform 0, 0.05 time.sleep delay raise last error class ConnectionPool: def init self, db path, pool size=5 : self.db path = db path self.pool size = pool size self. pool = def get self : if self. pool: return self. pool.pop conn = sqlite3.connect str self.db path , timeout=30 conn.execute "PRAGMA journal mode=WAL" conn.execute "PRAGMA busy timeout=30000" return conn def put self, conn : if len self. pool < self.pool size: self. pool.append conn else: conn.close pool = ConnectionPool DB PATH def log failure task: str, error: str, attempted fix: str, result: str, env: str = None : def insert : conn = pool.get try: conn.execute "INSERT INTO failures task, error, attempted fix, result, timestamp, env VALUES ?, ?, ?, ?, ?, ? ", task, error, attempted fix, result, time.time , env conn.commit finally: pool.put conn execute with retry insert Initialize on startup init db This is what I run across 3 concurrent agents. Zero "database is locked" errors since deployment. Have you dealt with SQLite "database is locked" errors in production? What's your retry strategy — exponential backoff, fixed intervals, or something else? I'm curious whether a connection pool actually helps or if it just adds complexity. Drop a comment below — I read every response.