The Batch API: Load Large Datasets into Agent Memory

Zep launched the Batch API for loading large datasets into agent memory, supporting batches up to 50,000 items with a progress dashboard and no impact on real-time ingestion. The API replaces earlier batch methods and is available to all Zep customers immediately.

The Batch API: Load Large Datasets into Agent Memory Zep's Batch API loads large datasets into agent memory faster, in batches up to 50,000 items, with a progress dashboard and no impact on real-time ingestion. Today we're shipping the Batch API , the new way to load large datasets into agent memory. It ingests data faster than sending the same operations one at a time, runs in batches of up to 50,000 items, tracks progress in a new dashboard, and stays separate from real-time ingestion. You create a batch, add your items across any graphs, users, and threads, then start processing. The Batch API replaces our earlier batch methods and is available to all Zep customers starting today. What the Batch API is The Batch API https://help.getzep.com/adding-batch-data?ref=blog.getzep.com is the recommended way to load large datasets into your Context Graphs https://help.getzep.com/graph-overview?ref=blog.getzep.com : a backfill, a migration from another system, a document collection, or any other bulk import. Instead of calling graph.add or thread.add messages once per item, you group everything into one batch and hand it to Zep as a single job. A batch follows a three-step lifecycle: create an empty batch, add items to it, then start processing. Items can be graph episodes or thread messages, and one batch can target any number of graphs, users, and threads. Why it matters The Batch API ingests large datasets faster than the same operations sent one at a time, and a single batch holds up to 50,000 items, enough to submit a full migration as one job. Its processing runs separately from real-time ingestion, so a big backfill won't affect ongoing ingestion. You also get visibility into how that work is going. The new batch dashboard shows every batch in your project: its status, a live progress bar, processed-item counts, and any per-item errors. When a batch completes, Zep can send a webhook https://help.getzep.com/webhooks?ref=blog.getzep.com batch-api-payload . How to use it Create a batch, add your items across one or more calls up to 500 per call, 50,000 per batch , then start processing. Zep returns immediately and ingests the batch asynchronously, grouping items by destination graph and processing them in the order you added them. Create a batch batch = client.batch.create metadata={"description": "Customer support backfill"} Add items: graph episodes and thread messages, any destination client.batch.add batch id=batch.batch id, items= BatchAddItem type="graph episode", user id="alice", data="Alice signed up for the Pro plan on 2024-06-15.", data type="text" , BatchAddItem type="thread message", thread id="alice support thread 42", content="My dashboard isn't loading.", role="user", name="Alice" , Start processing Zep ingests the batch asynchronously client.batch.process batch id=batch.batch id When you're loading historical data, set a created at timestamp on each item. Zep uses it to track when facts extracted from that data are valid or invalid. Getting started To get started, see the Batch API docs https://help.getzep.com/adding-batch-data?ref=blog.getzep.com . The Batch API is available to all Zep customers starting today. It replaces the older graph.add batch and thread.add messages batch https://help.getzep.com/adding-batch-data?ref=blog.getzep.com deprecated-batch-methods methods, which are now deprecated and will be removed in a future release.