We recently shipped an integration between Octic, our on-device AI recorder, and Octo, the open-source collaboration platform we use for agent-orchestrated work. The goal was straightforward: someone says something actionable in a meeting, and a Loop gets created and assigned without anyone lifting a finger.
This post walks through the pipeline end to end: audio capture, speech recognition, intent extraction, task creation, and agent execution.
Our team runs 6+ meetings a day. Decisions get made, action items get called out, and then... they die in someone's notebook. We tracked it for two weeks. Roughly 40% of verbally agreed tasks never made it into any tracking system. People just forgot to write them down, or wrote them down and forgot to transfer them.
We wanted to close that gap automatically.
The pipeline has five stages:
Mic → ASR (on-device) → NLU / intent extraction → Octo Loop creation → Agent execution
Each stage hands off a structured artifact to the next. No monolith, no single model doing everything.
Octic is a hardware recorder built by the same team behind the Lingting device, which was designed for noisy-environment speech capture. The key specs that matter for this pipeline:
The output is a timestamped, speaker-attributed transcript in JSON. Each segment looks roughly like:
{
"speaker": "fanrong",
"start_ms": 124500,
"end_ms": 131200,
"text": "Let's have the agent handle the weekly report generation, assign it to Elva's agent by Friday"
}
Privacy note: because ASR is on-device, we can use this in client meetings without worrying about data exfiltration. That was a hard requirement for us.
Raw transcripts are noisy. People repeat themselves, correct themselves mid-sentence, go on tangents. You can't just regex for "assign X to Y."
We run a lightweight NLU pass over the full transcript. The model extracts:
For action items specifically, we extract:
This runs as a post-meeting batch job. We experimented with real-time extraction during the meeting but found that waiting until the end produces much better results, because context from later in the conversation often clarifies earlier ambiguous statements.
This is where Octo comes in. Octo is an IM-native collaboration platform designed for human-agent work. The core abstraction is a Loop: a work unit that goes from conversation to delivery, with an owner, deliverables, and acceptance criteria.
For each extracted action item, we create a Loop via the Octo API:
The two-way link matters. When someone asks "why is my agent working on this?", the Loop shows the exact meeting moment where it was assigned.
Loops in Octo can be created two ways: manually through the UI, or via natural language. Our pipeline uses the API directly, but the natural language path is interesting for ad-hoc meeting follow-ups: you can literally type "create a loop for the quarterly review deck, assign to my agent" in the Workspace chat.
Here's where it gets interesting. In Octo, a Loop owner can be an Agent, not just a person. Agents are digital workforce clones: they inherit your authorizations, carry your preferences, and can autonomously pick up and execute assigned work.
When a Loop is created with an Agent as the owner:
Every acceptance or rejection gets stored as a Preference: a behavioral rule that the Agent references on future tasks. Over time, the Agent learns things like "this person prefers bullet points over paragraphs" or "always include the raw data table alongside the summary."
Rejected deliverables go back to the agent with specific feedback. The agent revises and resubmits. This creates an iterative cycle that's fully tracked in the Loop's timeline: brief, discussion, output, feedback, revision, acceptance.
A year from now, someone can open any Loop and see the full chain: what was said in the meeting, what task was created, what the agent produced, what got sent back, and what was finally accepted.
Speaker diarization quality is critical. Without reliable speaker attribution, you can't map "I'll handle this" to a specific person. Octic's diarization works well in rooms with 3-6 people, which covers most of our meetings.
Post-meeting batching beats real-time. Real-time intent extraction sounds cool in a demo but produces too many false positives in practice. People say things like "we should probably..." without meaning it as a commitment.
Acceptance criteria extraction is the hardest part. People rarely state explicit acceptance criteria in meetings. We default to a summary of the task context and let the Loop owner refine it before the agent starts work.
Preference accumulation is surprisingly useful. After about two weeks of active use, agents started producing first drafts that needed fewer revisions. The Preference system in Octo compounds over time.
Octo supports six orchestration modes for multi-agent collaboration: Solo, Roundtable, Critic, Pipeline, Split, and Swarm. For meeting-generated tasks, we mostly use Solo (single agent, simple task) and Pipeline (multi-step, ordered handoffs). The orchestration mode is selected based on task complexity at creation time.
We're still working on a few things:
If you're building something similar or have experience wiring ASR into structured workflows, I'd be interested in hearing what worked for you.
The Octo repo is at github.com/Mininglamp-OSS. The Loop and Agent systems are the most relevant parts for this use case.