{"slug": "i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory", "title": "I Built RecallixAI Around a Simple Problem: A Meeting Shouldn’t Reset Your Memory", "summary": "A developer built RecallixAI, a full-stack meeting intelligence system that captures Google Meet closed captions via a Chrome extension and turns transcripts into structured commitments, decisions, and open questions using Google Gemini, with persistent per-user meeting memory stored in Vectorize Hindsight. The system pairs a FastAPI backend with n8n, Google Calendar, Gmail, and Google Sheets automation, deliberately treating the transcript as an input rather than the product.", "body_md": "Most meeting software is very good at producing a transcript and surprisingly bad at answering the question I actually care about: what did we agree to last time? I built RecallixAI around that gap, using a live meeting pipeline for immediate context and Hindsight for the part that has to survive long after the browser tab is closed.\n\nWhat RecallixAI actually does\n\nRecallixAI is a full-stack meeting intelligence system built around four pieces:\n\nA Chrome companion that observes Google Meet closed captions without putting another bot into the meeting.\n\nA FastAPI backend that owns meeting sessions, transcripts, notes, analysis, and integrations.\n\nGoogle Gemini for turning a raw transcript plus private scratchpad into structured commitments, open questions, discrepancies, and a summary.\n\nVectorize Hindsight for persistent, per-user meeting memory, followed by automation through n8n, Google Calendar, Gmail, and Google Sheets.\n\nThe important distinction is that I don't treat the transcript as the product.\n\nThe transcript is an input. The useful state is the set of commitments, decisions, unresolved questions, people involved, and the history surrounding those things.\n\nThe repository reflects that separation. The backend exposes a streaming path for captions and notes, a completion path for post-meeting synthesis, and dedicated endpoints for meeting preparation and contact dossiers. The browser extension feeds the first path, while the dashboard consumes the resulting state.\n\nAt the top level, FastAPI mounts the API router and serves the dashboard:\n\napp = FastAPI(\n\n    title=\"Meeting Intelligence Agent API & Dashboard\",\n\n    description=\"Autonomous Meeting Intelligence & Prep Agent powered by Vectorize Hindsight & Google Calendar\",\n\n    version=\"2.0.0\"\n\n)\n\napp.include_router(api_router, prefix=\"/api\")\n\nThat sounds ordinary, and intentionally so. I wanted the orchestration to live behind a small HTTP surface instead of spreading meeting state across browser code and third-party services.\n\nThe interesting part starts before the LLM\n\nI made a deliberate choice not to build the capture layer around a meeting bot.\n\nThe Chrome extension watches the closed-caption DOM inside Google Meet. It uses MutationObserver to detect changes, extracts the speaker and text, and sends structured caption events to the backend.\n\nThe core loop is small:\n\nasync function pushCaptionToBackend(speaker, text) {\n\n  const meetingId = getMeetingId();\n\n  const payload = {\n\n    meeting_id: meetingId,\n\n    speaker: speaker || \"Participant\",\n\n    text: text,\n\n    timestamp: new Date().toISOString()\n\n  };\n\nawait fetch(`${API_BASE_URL}/api/stream/caption`, {\n\n    method: \"POST\",\n\n    headers: { \"Content-Type\": \"application/json\" },\n\n    body: JSON.stringify(payload)\n\n  });\n\n}\n\nThe extension is deliberately dumb. It doesn't try to summarize the meeting, infer commitments, or maintain a second copy of the application's business logic.\n\nThat matters because browser integrations are brittle enough already. Google Meet's DOM can change. I would rather have one small responsibility fail than have meeting capture, inference, memory, and workflow automation coupled to selectors in a content script.\n\nThere is also a practical detail that ended up being important: captions can be emitted incrementally and sometimes repeat. The extension tracks the previous caption and speaker, while the backend also avoids consecutive duplicate caption records.\n\nThe backend receives those events through a simple endpoint:\n\n[@router](https://dev.to/router).post(\"/stream/caption\")\n\ndef ingest_stream_caption(req: StreamCaptionRequest):\n\n    caption = session_service.add_caption(\n\n        meeting_id=req.meeting_id,\n\n        speaker=req.speaker,\n\n        text=req.text,\n\n        timestamp=req.timestamp\n\n    )\n\n```\nreturn {\n    \"status\": \"ingested\",\n    \"meeting_id\": req.meeting_id,\n    \"caption\": caption\n}\n```\n\nThat gives me a clean boundary: capture produces events; the backend owns meeting state.\n\nI kept live state and long-term memory separate\n\nThis was probably the most important architectural decision.\n\nDuring a call, I need low-friction state. I need the latest captions and the user's scratchpad immediately. I don't need a sophisticated memory system for every keystroke.\n\nThe session service therefore keeps an active MeetingSession with captions, notes, attendees, timestamps, and status.\n\nclass MeetingSession:\n\n    def **init**(self, meeting_id: str, title: Optional[str] = None):\n\n        self.meeting_id = meeting_id\n\n        self.title = title or f\"Meeting {meeting_id}\"\n\n        self.start_time = datetime.datetime.now(datetime.timezone.utc).isoformat()\n\n        self.captions = []\n\n        self.user_notes = \"\"\n\n        self.attendees = []\n\n        self.status = \"active\"\n\nHindsight enters at a different boundary.\n\nWhen a meeting is complete, Gemini reconciles the spoken transcript with the user's scratchpad. The resulting structured analysis is retained in Hindsight as durable memory.\n\nThat gives me two very different stores:\n\nSession state: what is happening right now.\n\nHindsight memory: what should still matter weeks later.\n\nI don't want a database of raw events masquerading as memory. I want the system to be able to answer a future question in terms of the relationship and its history.\n\nThat is where Hindsight on GitHub fits naturally into the design. Hindsight provides retain, recall, and reflection-oriented memory primitives rather than forcing me to build a retrieval layer from scratch. Its model is specifically aimed at persistent agent memory rather than simply storing conversation history.\n\nHindsight is useful because the question is contextual\n\nA conventional search over old transcripts can answer \"find meetings containing pricing.\"\n\nIt is much less useful for:\n\n\"I'm meeting Sarah again. What did we promise each other, and what was still unresolved?\"\n\nRecallixAI sends Hindsight a query shaped around that actual task:\n\nquery = f\"What was discussed, promised, or left unresolved with {attendee_email}?\"\n\nrecall_response = self.client.recall(\n\n    bank_id=bank_id,\n\n    query=query,\n\n    budget=\"mid\"\n\n)\n\nThe memory bank is scoped to the user, and the bank gets a mission describing what matters:\n\nMISSION_STATEMENT = (\n\n    \"Track all meeting dialogues, commitments made by both parties, \"\n\n    \"agreed deadlines, unresolved questions, and user note preferences.\"\n\n)\n\nI also configure Hindsight's disposition for this workload:\n\nDEFAULT_DISPOSITION = {\n\n    \"literalism\": 4,\n\n    \"skepticism\": 2,\n\n    \"empathy\": 3\n\n}\n\nI like this because memory is not completely generic. A meeting assistant has a different definition of useful memory from a coding assistant or a personal chatbot.\n\nThe Hindsight documentation describes this broader model through memory banks, retain, recall, reflect, observations, and consolidated knowledge. For RecallixAI, the practical benefit is that I can treat Hindsight as a dedicated long-term memory layer instead of inventing an application-specific combination of embeddings, metadata filters, and transcript search.\n\nThis is also why the distinction between agent memory and ordinary retrieval matters. The system isn't merely asking, \"Which old text looks similar to this query?\" It is trying to preserve useful knowledge about an ongoing relationship.\n\nThe LLM isn't allowed to define the application's state\n\nAnother choice I made was to force the meeting analysis into a typed structure.\n\nGemini receives the transcript and scratchpad, but the output has to conform to MeetingAnalysis:\n\nclass MeetingAnalysis(BaseModel):\n\n    summary: str\n\n    promises_by_us: List[str] = Field(default_factory=list)\n\n    promises_by_them: List[str] = Field(default_factory=list)\n\n    missed_or_pending_followups: List[str] = Field(default_factory=list)\n\n    note_discrepancies: List[str] = Field(default_factory=list)\n\n    suggested_followup_date: Optional[str] = None\n\nThe model call requests JSON with that schema:\n\nresponse = self.gemini_client.models.generate_content(\n\n    model=settings.GEMINI_MODEL,\n\n    contents=prompt,\n\n    config={\n\n        \"response_mime_type\": \"application/json\",\n\n        \"response_schema\": MeetingAnalysis\n\n    }\n\n)\n\nThat gives the rest of the application something much more reliable than a generated paragraph.\n\nAfter synthesis, the backend builds a durable memory record containing the summary, commitments from both sides, pending follow-ups, discrepancies, and the original notes. It then calls Hindsight's retain() with meeting metadata.\n\nself.client.retain(\n\n    bank_id=bank_id,\n\n    content=content,\n\n    context=context,\n\n    document_id=doc_id,\n\n    metadata=metadata\n\n)\n\nThe interesting part is what happens next. The same memory is available when preparing for another meeting:\n\nresult = hindsight_service.recall_prep_context(\n\n    user_id=req.user_id,\n\n    attendee_email=req.attendee_email\n\n)\n\nSo the workflow becomes:\n\ncapture → reconcile → retain → recall → act\n\nThat loop is the actual system.\n\nA concrete interaction\n\nImagine a meeting where I say:\n\nI'll send the enterprise pricing breakdown by Thursday.\n\nThe client says they'll send their security questionnaire the next morning.\n\nDuring the call, I write:\n\n[Promise: I will deliver the updated enterprise pricing breakdown by Friday afternoon]\n\n[Action: Sarah needs to send over technical security questionnaire]\n\nThe transcript says Thursday, while my note says Friday.\n\nRecallixAI doesn't simply preserve both pieces of text and call the job done. Gemini is instructed to compare the two sources and produce a discrepancy.\n\nThe analysis can therefore contain:\n\nMy commitment: send the pricing breakdown.\n\nTheir commitment: send the security questionnaire.\n\nAn unresolved item: clarify the remaining security/SLA questions.\n\nA discrepancy: my scratchpad says Friday while the spoken agreement says Thursday.\n\nA proposed follow-up date.\n\nThat structured result is retained in Hindsight.\n\nWhen the next meeting appears on the calendar, the preparation endpoint asks Hindsight what was previously discussed, promised, or left unresolved. The dashboard can then present the relationship context before I join the call.\n\nAfter the meeting is completed, action items can also be sent through n8n. The backend turns extracted commitments into task records and posts them to the configured workflow. A separate calendar endpoint can create the confirmed follow-up event.\n\nThis separation is useful because n8n is handling integration plumbing, not business reasoning. Gemini determines what the meeting produced. Hindsight preserves why it matters later. n8n handles the mechanical work of moving those results into external systems.\n\nWhat I learned building it\n\nMy first instinct with meeting software would have been to store everything and add search later.\n\nThat is backwards.\n\nThe useful unit is not \"a transcript from September 29.\" It is \"what matters when I talk to this person again?\" Defining that question early makes the memory mission, metadata, retention content, and recall query much easier to reason about.\n\nA live session wants simple mutable state.\n\nLong-term memory wants durable, searchable knowledge.\n\nTrying to make one mechanism handle both creates unnecessary complexity. Keeping the session service lightweight and using Hindsight at the retention boundary made the architecture easier to reason about.\n\nI don't want downstream code parsing prose from a model.\n\nThe MeetingAnalysis schema gives the model freedom to infer the content while keeping the application's state predictable. That makes persistence, automation, UI rendering, and testing much easier.\n\nSending every caption independently into long-term memory would produce a lot of noise.\n\nRecallixAI instead synthesizes the meeting first and retains a compact record containing the things I actually expect to matter later: commitments, unresolved work, discrepancies, decisions, and context.\n\nHindsight then has a better-quality memory to recall from.\n\nScheduling a follow-up or writing a spreadsheet row is easy.\n\nKnowing why a follow-up is necessary is harder.\n\nI kept that ordering explicit: first capture the conversation, then analyze it, then retain the durable result, and only then trigger external actions.\n\nThe bigger lesson\n\nThe interesting part of RecallixAI isn't transcription, summarization, or calendar automation individually. None of those are particularly new problems.\n\nThe hard part is continuity.\n\nA meeting produces decisions today. Those decisions need to influence what I do tomorrow. The next conversation should not begin with a blank context window and a pile of old transcripts.\n\nThat is why Hindsight became a central architectural component rather than an optional search feature. It gives RecallixAI a place to retain the parts of a meeting that should survive the meeting itself, and a way to retrieve that knowledge in the context of the next interaction.\n\nThe resulting architecture is intentionally straightforward:\n\nGoogle Meet captions → FastAPI session state → Gemini reconciliation → Hindsight memory → pre-meeting recall → external actions.\n\nThat is the pattern I would reuse elsewhere: keep the real-time path simple, make the LLM output structured, and treat long-term memory as its own system with a clear purpose.\n\nThe goal isn't for the application to remember everything.\n\nIt's for it to remember the things that will change what happens next.", "url": "https://wpnews.pro/news/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory", "canonical_source": "https://dev.to/vijay_siddharth_d372ab215/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory-46ml", "published_at": "2026-09-29 18:04:55+00:00", "updated_at": "2026-09-29 18:16:53.421129+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "large-language-models", "ai-products", "developer-tools"], "entities": ["RecallixAI", "Google Meet", "Google Gemini", "Vectorize Hindsight", "FastAPI", "n8n", "Google Calendar", "Gmail"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory", "markdown": "https://wpnews.pro/news/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory.md", "text": "https://wpnews.pro/news/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory.txt", "jsonld": "https://wpnews.pro/news/i-built-recallixai-around-a-simple-problem-a-meeting-shouldnt-reset-your-memory.jsonld"}}