Escaping the Stateless Trap: Building a Context-Aware Support Agent

A developer built IRIS (Intelligent Recall & Issue Support), a multi-tenant API service that turns e-commerce systems into context-aware support agents. The system uses a two-layer memory architecture with Hindsight to maintain long-term state across sessions, avoiding the common problem of chatbots forgetting past interactions. It integrates with Groq-hosted llama3-70b-8192 for fast inference and connectors like Shopify for real-time order data.

Most people think building a support chatbot is about generating human-like text. It isn’t. The real problem is memory. Or more specifically, the complete lack of it. I got tired of prompt engineering hacks and started looking for something better. I didn’t want a bot that sounded smart for one message and then immediately forgot everything like it hit its head on a table. That’s how IRIS Intelligent Recall & Issue Support started. Instead of building yet another chatbot, I built a multi-tenant API service that plugs into existing e-commerce systems and turns them into context-aware support agents. Because apparently, remembering what a customer said five minutes ago is still considered advanced technology. Most bots I tested had the same fatal flaw: every interaction felt like a first date. A customer complains three times about a delayed package, and the bot still asks for the order number again like it has goldfish-level cognition. Not ideal. So the goal became simple: give the agent memory across sessions, not just within a single chat window. What IRIS Does And Why It’s Actually Useful IRIS is a stateful API layer built with FastAPI that sits between: The customer chat interface Order Management Systems OMS A large language model Its job is to route messages, manage multi-tenant authentication, and most importantly, maintain long-term state. The system follows an integration-first approach. Businesses don’t want another dashboard. They want something that quietly works in the background without forcing their team to learn yet another UI. So IRIS is headless. One API. Multiple tenants. Strict data isolation per brand. The architecture is built around three core pillars: We use Groq-hosted llama3-70b-8192 for fast inference. Speed matters because the system performs multiple internal steps before responding. Connectors like Shopify and REST APIs fetch real-time order data. This prevents hallucinations like “your package is delivered” when it’s still sitting in a warehouse. Instead of stuffing entire chat histories into a vector database or relying on massive context windows, I used Hindsight for structured agent memory. How the System Actually Works When a message comes in: The system queries the memory layer for: Customer history Behavioral profile It fetches live order data from OMS. It checks a global incident stream: Are multiple users reporting the same issue? All of this context is compiled into a structured prompt. The LLM generates a response. If an action is required like a refund , it outputs structured JSON. The backend extracts that JSON, executes the action, and removes it from the user-visible response. The interaction is stored back into memory. Simple idea. Annoyingly complex execution. The Real Challenge: Memory Architecture The hardest part wasn’t calling an LLM API. That’s easy. The real challenge was structuring memory so the system could be: Personalized per user Aware of system-wide issues Initially, I tried dumping everything into one vector store per tenant. That went about as well as you’d expect. The agent started mixing user contexts. User A’s complaint showed up in User B’s conversation. Not great unless you enjoy chaos. So I switched to a segmented memory model using Hindsight. Two-Layer Memory Design Each user gets their own memory space: Communication style Past issues Preferences Tenant-wide memory that tracks trends: Issue spikes System outages Common failures This separation completely changed the system behavior. It stopped reacting and started anticipating. Code Walkthrough Message Processing async def process message tenant id: str, user id: str, message: str : 1. Detect if this is a systemic issue issue type = detect issue type message if issue type: await report to global memory tenant id, issue type Check if we are currently in an active incident for this issue active incident = await check active incidents tenant id, issue type if active incident: Short-circuit standard troubleshooting context "incident alert" = f"Known issue: {active incident.description}" 2. Recall personal history customer history = await memory client.recall bank id=f"{tenant id} user {user id}", query=message, limit=5 3. Generate a quick reflection on the customer's state profile = await memory client.reflect bank id=f"{tenant id} user {user id}" return await generate llm response message, customer history, profile, context The key part here is reflect. Instead of dumping raw chat logs into the LLM, we generate a summarized profile. This reduces token usage and improves accuracy. Storing Interactions async def retain interaction tenant id: str, user id: str, user msg: str, agent response: str : bank id = f"{tenant id} user {user id}" Store the interaction in the customer's specific memory bank await memory client.retain bank id=bank id, content=f"Customer: {user msg}\nAgent: {agent response}", metadata={"timestamp": get current time } Nothing fancy here. Just disciplined storage instead of hoping the LLM remembers things out of kindness. Action Execution def extract and execute action llm response: str, order data: dict : Look for a JSON block at the end of the response match = re.search r' ', llm response, re.DOTALL if match: try: action req = json.loads match.group 1 if action req.get "action" == "initiate refund": execute refund action req.get "order id" Strip the JSON so the user doesn't see it clean response = llm response.replace match.group 0 , "" .strip return clean response, True except json.JSONDecodeError: pass return llm response, False Instead of trusting the LLM to behave, we force it into a structured output format and clean things up afterward. Because optimism is not a system design strategy. Results: Before vs After Without memory: User: Where is my replacement? Bot: Please provide your order number. With IRIS: User: Where is my replacement? IRIS: I see we initiated a replacement for order 12345 yesterday due to damage. It shipped this morning via UPS Tracking: 1Z9999 and should arrive by Thursday. That’s the difference between a chatbot and a support system. Incident Detection in Action During a simulated outage: First few users → normal troubleshooting After pattern detection → immediate acknowledgment Response shifts to: “We’re currently experiencing checkout issues. Our team is working on it. I’ll notify you once it’s resolved.” This avoids hundreds of duplicate support tickets and reduces system load. Lessons Learned 1. State Intelligence LLMs are smart, but without memory they’re basically polite amnesiacs. Memory must be a core architectural component, not an afterthought. 2. Summarize, Don’t Dump Raw transcripts degrade performance. Reflection-based summaries improve both accuracy and efficiency. 3. Separate Responsibilities Don’t ask the LLM to generate text and execute logic perfectly at the same time. Use structured outputs and backend validation. 4. Speed Matters Fast inference Groq makes multi-step workflows viable. Slow systems can’t afford layered reasoning. Final Thought Building a stateful agent isn’t about crafting the perfect prompt. It’s about building the infrastructure that ensures the model always receives the right context at the right time. Everything else is just clever wording wrapped around forgetfulness.