One Stream, Three Backends: Streaming FFmpeg to Local, Azure, and R2 with Python Storix, a Python storage library, enables streaming data from producers like FFmpeg to multiple backends—local files, Azure Blob Storage, and Cloudflare R2—without provider-specific code. The library's developer, who built Storix around native Python types, demonstrates how an async generator feeding an AsyncIterator[bytes] can be written to any configured backend using a single echo() call, eliminating temporary files and SDK branching. Why Storix uses native Python streams, provider-agnostic storage sessions, and composable layers instead of provider-specific upload code. A process is already producing data. It might be FFmpeg generating media, a compressor writing an archive, an HTTP request delivering an upload, a database exporting records, or an inference pipeline producing artifacts. The storage destination should not determine how that producer works. A common workflow looks like this: php producer - write a temporary file - reopen the file - upload it through a provider-specific SDK - delete the temporary file That works, but it spreads storage concerns into the producer and creates an intermediate file that may not need to exist. I wanted the flow to look like this instead: php producer - Iterable bytes or AsyncIterable bytes - Storix - configured storage backend In this demo, FFmpeg generates a fragmented MP4 through stdout. Python exposes that output as an AsyncIterator bytes , and Storix writes the same stream to: The Python code and logical destination path stay fixed. Only the selected Storix configuration changes. The demo uses three destinations to keep the sequence short and readable. Storix also supports Azure Data Lake Gen2, Amazon S3 and compatible stores such as MinIO, and Google Cloud Storage. This is the part of the demo that matters: python from storix.aio import get storage async with get storage as fs: await fs.mkdir "/launch", parents=True await fs.echo ffmpeg stream , "/launch/one-stream-three-backends.mp4", chunk size=4 1024 1024, There is no Azure SDK, S3 SDK, or local filesystem branch in the application logic. There is also no application-managed temporary video file. FFmpeg produces chunks. Storix consumes them. Developer experience is one of the main reasons I built Storix around native Python types. You should not have to convert data into a library-specific upload object before it can be stored. echo accepts the values Python developers already work with: str , bytes , bytearray , and other buffer-compatible objects IO str and IO bytes , including regular open ... file objects Iterable str | Buffer and Iterable bytes | Buffer AsyncIterable str | Buffer and AsyncIterable bytes | Buffer through storix.aio The public contract is built from standard Python IO , Iterable , AsyncIterable , and buffer-protocol types rather than a Storix-specific stream class. The FFmpeg producer is therefore an ordinary async generator: python import asyncio import contextlib from collections.abc import AsyncIterator async def ffmpeg stream - AsyncIterator bytes : process = await asyncio.create subprocess exec command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, assert process.stdout is not None assert process.stderr is not None Drain stderr while stdout is being consumed so FFmpeg cannot block on a full error pipe. The complete sample also reports FFmpeg failures. stderr task = asyncio.create task process.stderr.read try: while chunk := await process.stdout.read 64 1024 : yield chunk return code = await process.wait stderr = await stderr task if return code = 0: detail = stderr.decode "utf-8", errors="replace" .strip raise RuntimeError detail or f"FFmpeg exited with status {return code}" finally: if process.returncode is None: process.kill await process.wait if not stderr task.done : stderr task.cancel with contextlib.suppress asyncio.CancelledError : await stderr task Nothing in this function knows that Storix exists. It could feed those chunks into an HTTP response, a message broker, a hashing pipeline, a parser, or any other consumer that accepts an async iterable. Storix is only the destination. See the echo reference https://storix.mghalix.com/reference/storix/ echo for the complete input contract. Not every workflow begins with a subprocess. A regular file opened through Python can be passed directly: python from storix.aio import get storage async with get storage "local", base="./data" as fs: await fs.mkdir "/reports", parents=True with open "report.parquet", "rb" as source: await fs.echo source, "/reports/report.parquet" Text files work the same way: async with get storage as fs: await fs.mkdir "/events", parents=True with open "events.ndjson", encoding="utf-8" as source: await fs.echo source, "/events/events.ndjson" You can also produce chunks yourself: php from collections.abc import AsyncIterator async def generate export - AsyncIterator bytes : async for row in database rows : yield encode row row async with get storage as fs: await fs.mkdir "/exports", parents=True await fs.echo generate export , "/exports/customers.ndjson", The storage API does not force the producer to become storage-aware. Writing is only half of the flow. Storix can read files incrementally with stream . The synchronous API produces a regular iterator: python from storix import get storage with get storage "local", base="./data" as fs: for chunk in fs.stream "/videos/source.mp4" : downstream.send chunk The asynchronous API produces an async iterator: python from storix.aio import get storage async with get storage as fs: async for chunk in fs.stream "/videos/source.mp4", chunk size=64 1024, : await downstream.send chunk The downstream consumer might be an HTTP response, decompressor, parser, media processor, hashing pipeline, inference component, or another storage destination. For small, known-size files, cat returns the complete contents as bytes . For larger workloads, stream lets the application process data incrementally instead of materializing the complete object first. For downloads into a seekable file, Storix 0.5.0 can go further than an ordered stream. download may fetch several byte ranges of one large object concurrently and write each range at its destination offset. stream remains the ordered incremental API for arbitrary consumers. In one measured 200 MiB Azure download over a home connection, eight ranges reduced wall time from 61.53 seconds to 25.51 seconds, with peak RSS of 173 MB. This is one measurement, not a universal speed guarantee. Each range is a separate request, so the throughput improvement trades against transaction count. Use ranges=1 , or STORIX MAX TRANSFER RANGES=1 , to keep every download on one stream. See: The demo uses get storage without naming a provider in the Python code: python from storix.aio import get storage async with get storage as fs: ... The active provider can come from environment configuration: STORIX PROVIDER=local uv run python demo.py STORIX PROVIDER=azure uv run python demo.py STORIX PROVIDER=s3 uv run python demo.py Provider-specific settings remain namespaced. For Azure: STORIX PROVIDER=azure STORIX AZURE CONTAINER=storix-demo STORIX AZURE ACCOUNT NAME=my-account STORIX AZURE CREDENTIAL=... Optional. The default is auto. STORIX AZURE KIND=blob With kind="auto" , Storix checks whether the account has hierarchical namespaces enabled and selects either the ADLS Gen2 backend or the Blob backend. Explicit blob or adls selection skips that detection when the intended surface is already known or account-level detection is unavailable. For Cloudflare R2: STORIX PROVIDER=s3 STORIX S3 BUCKET=storix-demo STORIX S3 REGION=auto STORIX S3 ENDPOINT=https://