{"slug": "show-hn-a-durable-filesystem-layer-for-ai-agents", "title": "Show HN: A durable filesystem layer for AI agents", "summary": "CelestoAI released SmolFS, an open-source durable filesystem layer for AI agents that persists workspace data across agent process restarts. The tool mounts as a standard directory, supports local and cloud storage backends including Redis and S3, and provides Python and TypeScript SDKs wrapping a Rust core. SmolFS aims to simplify state management for short-lived agent runtimes by decoupling storage setup from agent code.", "body_md": "SmolFS gives agents a workspace folder that can survive after the agent process stops. You mount it like a normal directory, write files into it, unmount it when the job is done, and mount it again later from another process.\n\nSmolFS owns the developer experience around creating volumes, checking the machine, mounting, flushing, unmounting, and inspecting status. The storage backend is installed and managed for you.\n\n|\nAgents can keep files across short-lived runtimes without each runtime managing storage setup directly. |\nUse |\n|\nUse Redis plus S3-compatible object storage when the same workspace needs to outlive one machine. |\nRun |\n|\nPython and TypeScript bindings call the same Rust core so agent tools can use SmolFS without shelling out. |\nCloud metadata, buckets, and credentials stay explicit so durable agent data is easier to audit. |\n\n**Keep agent work across turns.** Mount the same workspace again after an agent process exits.**Share a workspace across runtimes.** Put metadata in Redis and file contents in S3-compatible storage.**Test locally before using cloud storage.** Start with`--dev`\n\n, then switch to explicit metadata and object storage settings.**Wrap storage in agent tooling.** Use the Python or TypeScript SDK from an agent runner instead of teaching every agent process about storage internals.\n\nInstall SmolFS:\n\n```\ncurl -fsSL https://raw.githubusercontent.com/CelestoAI/smolfs/main/scripts/install.sh | sh\n```\n\nThe installer downloads the latest CLI release asset for your platform and\ninstalls SmolFS' managed storage backend. If no release asset exists yet, use the\nsource checkout flow in [Development](#development). Set\n`SMOLFS_INSTALL_BACKEND=0`\n\nif you only want to install the CLI. Set\n`SMOLFS_VERSION=dev`\n\nto try the latest successful build from `main`\n\n; tagged\n`v*`\n\nreleases remain the stable channel.\n\nCheck the machine and try a local volume:\n\n```\nsmolfs doctor\nsmolfs init demo --dev\nsmolfs mount demo ./workspace\necho hello > ./workspace/hello.txt\nsmolfs flush demo\nsmolfs unmount demo\nsmolfs mount demo ./workspace\ncat ./workspace/hello.txt\n```\n\nSmolFS needs local mount support on the machine that mounts volumes. Mount support lets SmolFS provide a folder your tools can read and write.\n\n## Install the Python SDK with the CLI\n\n```\ncurl -fsSL https://raw.githubusercontent.com/CelestoAI/smolfs/main/scripts/install.sh | SMOLFS_INSTALL_PYTHON=1 sh\n```\n\nThe installer runs `uv add smolfs`\n\nfrom a directory with `pyproject.toml`\n\n, or\n`uv pip install smolfs`\n\ninside an active virtualenv. Set\n`SMOLFS_PYTHON_MODE=user`\n\nto use `uv pip install --user smolfs`\n\n.\n\nThe normal SmolFS flow is: check the machine, create a volume, mount it, use the directory, flush important writes, unmount it, and mount it again when you need the same files later.\n\nRun `doctor`\n\nbefore creating or mounting volumes:\n\n```\nsmolfs doctor\n```\n\n`doctor`\n\nchecks whether SmolFS has its managed storage backend and whether the\nmachine can mount local directories.\n\nUseful options:\n\n`smolfs doctor --install`\n\ninstalls SmolFS' managed storage backend.`smolfs doctor --json`\n\nprints the same report as JSON for scripts.\n\nSmolFS looks for its home directory in `SMOLFS_HOME`\n\n. If it is not set, SmolFS\nuses `~/.smolfs`\n\n. The home directory stores SmolFS config, volume records, logs,\nmanaged binaries, and local dev-volume data.\n\nA volume is the named workspace that SmolFS can mount later.\n\n```\nsmolfs init demo --dev\n```\n\n`--dev`\n\ncreates a local-only volume. It uses SQLite for metadata and local files\nfor object data under the SmolFS home directory. This is the simplest path for\ntrying SmolFS on one machine.\n\nCloud volumes need explicit metadata and object storage settings. Metadata stores the file tree. Object storage is where file contents live.\n\n```\nsmolfs init agent-workspace \\\n  --metadata redis://localhost:6379/1 \\\n  --storage s3 \\\n  --bucket https://my-bucket.s3.us-east-2.amazonaws.com\n```\n\nYou can pass object storage in either of these forms:\n\n`--store s3://bucket/prefix`\n\n,`--store gs://bucket/prefix`\n\n, or`--store file:///path/to/objects`\n\n.`--storage TYPE --bucket BUCKET`\n\n, which is useful for S3-compatible services that expect an endpoint-style bucket URL.\n\nFor Cloudflare R2 or another S3-compatible service, keep credentials in the environment used by SmolFS. Do not put access keys in command arguments or logs.\n\n```\nset -a\n. ./.env\nset +a\n\nexport SMOLFS_HOME=/tmp/smolfs-r2-home\nVOL=\"r2demo-$(date +%Y%m%d%H%M%S)\"\n\nsmolfs init \"$VOL\" \\\n  --metadata \"$SMOLFS_R2_METADATA\" \\\n  --storage s3 \\\n  --bucket \"$SMOLFS_R2_BUCKET_URL\"\n```\n\nMounting makes the volume appear as a normal local directory:\n\n```\nsmolfs mount demo ./workspace\necho hello > ./workspace/hello.txt\ncat ./workspace/hello.txt\n```\n\nSmolFS creates the mount directory if it does not exist. After the mount succeeds, programs can read and write files through that directory.\n\nUseful options:\n\n`--check-storage`\n\nasks SmolFS to test object storage access before the mount completes.`--foreground`\n\nruns the mount process in the foreground instead of starting it in the background.\n\nRun `flush`\n\nwhen you want a best-effort check that recent writes have reached\nthe mounted filesystem:\n\n```\nsmolfs flush demo\n```\n\nRun `status`\n\nto see known volumes:\n\n```\nsmolfs status\nsmolfs status demo\nsmolfs status --json\n```\n\nUnmount when the job is done:\n\n```\nsmolfs unmount demo\n```\n\n`unmount`\n\nasks SmolFS to flush before detaching the mountpoint. Use\n`smolfs umount demo`\n\nif you prefer the shorter alias. Add `--force`\n\nwhen the\nmountpoint is busy and you want SmolFS to force the unmount.\n\nAfter unmounting, you can mount the same volume again and read the files:\n\n```\nsmolfs mount demo ./workspace\ncat ./workspace/hello.txt\n```\n\n| Command | What it does |\n|---|---|\n`smolfs doctor` |\nChecks SmolFS storage, local mount support, and configuration. |\n`smolfs init NAME --dev` |\nCreates a local development volume. |\n`smolfs init NAME --metadata URL --storage TYPE --bucket BUCKET` |\nCreates a cloud volume with explicit metadata and object storage. |\n`smolfs mount NAME PATH` |\nMounts a volume at a local directory. |\n`smolfs flush NAME` |\nProbes the mounted volume and syncs a small file through it. |\n`smolfs status [NAME]` |\nShows known volumes and current mountpoints. |\n`smolfs unmount NAME` |\nUnmounts a mounted volume and asks SmolFS to flush. |\n`smolfs umount NAME` |\nAlias for `smolfs unmount NAME` . |\n\nEvery command has its own help page:\n\n```\nsmolfs help\nsmolfs init --help\n```\n\nThe Python package is SDK-only. Install it with `uv`\n\n:\n\n```\nuv add smolfs\n```\n\nFor local development from this checkout:\n\n```\nuv run --isolated --with-editable ./bindings/python python -c \"from smolfs import doctor; print(doctor())\"\n```\n\nUse the SDK from any Python agent runner:\n\n``` python\nfrom pathlib import Path\n\nfrom smolfs import SmolFS, doctor\n\nreport = doctor()\nif not report[\"storage_backend\"][\"found\"] or not report[\"mount_support\"][\"found\"]:\n    raise RuntimeError(f\"SmolFS is not ready: {report}\")\n\nfs = SmolFS.from_env()\nvolume = fs.ensure_volume(\"demo\", dev=True)\nmount = fs.mount(volume.name, \"./workspace\")\n\nworkspace = Path(mount.mountpoint)\n(workspace / \"hello.txt\").write_text(\"hello from SmolFS\\n\")\n\ntry:\n    fs.flush(volume.name)\nfinally:\n    fs.unmount(volume.name)\n```\n\nCloud volumes use the same API with explicit metadata and object storage:\n\n```\nfs.ensure_volume(\n    \"agent-workspace\",\n    metadata=\"redis://localhost:6379/1\",\n    storage=\"s3\",\n    bucket=\"https://my-bucket.s3.us-east-2.amazonaws.com\",\n)\n```\n\nFor S3-compatible services such as MinIO or Cloudflare R2, pass the service\nbucket URL and provide `ACCESS_KEY`\n\nand `SECRET_KEY`\n\nin the environment used by\nSmolFS.\n\nThe TypeScript package is a native Node.js binding over the same Rust core. The npm package ships prebuilt native bindings for Linux and macOS on x86_64 and arm64.\n\nInstall it with npm:\n\n```\nnpm install @celestoai/smolfs\n```\n\nFor local development from this checkout, use Node.js 18 or newer:\n\n```\ncd bindings/node\nnpm ci\nnpm run build:debug\nnpm test\n```\n\nUse the SDK from a Node.js agent runner:\n\n``` js\nimport { SmolFS, doctor } from \"@celestoai/smolfs\";\nimport { writeFile } from \"node:fs/promises\";\nimport { join } from \"node:path\";\n\nconst report = doctor();\nif (!report.storageBackend.found || !report.mountSupport.found) {\n  throw new Error(`SmolFS is not ready: ${JSON.stringify(report)}`);\n}\n\nconst fs = SmolFS.fromEnv();\nconst volume = fs.ensureVolume({ name: \"demo\", dev: true });\nconst mount = fs.mount({ name: volume.name, path: \"./workspace\" });\n\ntry {\n  await writeFile(join(mount.mountpoint, \"hello.txt\"), \"hello from SmolFS\\n\");\n  fs.flush(volume.name);\n} finally {\n  fs.unmount(volume.name);\n}\n```\n\nCloud volumes use the same options object:\n\n```\nfs.ensureVolume({\n  name: \"agent-workspace\",\n  metadata: \"redis://localhost:6379/1\",\n  storage: \"s3\",\n  bucket: \"https://my-bucket.s3.us-east-2.amazonaws.com\"\n});\n```\n\nWork from a source checkout when you are changing SmolFS itself or when a CLI release asset has not been published yet.\n\nBuild and check the CLI:\n\n```\ncargo build -p smolfs-cli\n./target/debug/smolfs doctor\n```\n\nRun the normal quality checks:\n\n```\ncargo fmt --all -- --check\ncargo clippy --workspace -- -D warnings\ncargo test --workspace\n```\n\nRun the R2-style lifecycle from this checkout:\n\n```\ncargo build -p smolfs-cli\n\nset -a\n. ./.env\nset +a\n\nexport SMOLFS_HOME=/tmp/smolfs-r2-home\nVOL=\"r2demo-$(date +%Y%m%d%H%M%S)\"\nMOUNT=\"/tmp/smolfs-r2-workspace\"\n\n./target/debug/smolfs init \"$VOL\" \\\n  --metadata \"$SMOLFS_R2_METADATA\" \\\n  --storage s3 \\\n  --bucket \"$SMOLFS_R2_BUCKET_URL\"\n\n./target/debug/smolfs mount \"$VOL\" \"$MOUNT\"\necho \"hello from smolfs r2\" > \"$MOUNT/hello.txt\"\n./target/debug/smolfs flush \"$VOL\"\n./target/debug/smolfs unmount \"$VOL\"\n./target/debug/smolfs mount \"$VOL\" \"$MOUNT\"\ncat \"$MOUNT/hello.txt\"\n```\n\nRun the MinIO integration test path when the SmolFS storage backend, Redis, and a MinIO bucket are available:\n\n```\nSMOLFS_RUN_INTEGRATION=1 cargo test --workspace -- --nocapture\n```\n\nBuild the Python wheel:\n\n```\nuvx maturin build --manifest-path bindings/python/Cargo.toml --interpreter python\n```\n\nDevelop the Python binding locally:\n\n```\nuvx maturin develop --manifest-path bindings/python/Cargo.toml\n```\n\nTest the TypeScript SDK:\n\n```\ncd bindings/node\nnpm ci\nnpm test\n```\n\nSmolFS stores agent workspace data outside the sandbox lifecycle. Treat it like durable infrastructure.\n\n- Do not log credentials, S3 access keys, Redis URLs with secrets, or mount tokens.\n- Prefer explicit object-store configuration over hidden global state.\n- Make mount and unmount behavior idempotent where possible.\n- Fail loudly on missing storage backend, missing metadata URLs, missing object-store config, or missing local mount support.\n- Avoid changes that weaken persistence guarantees without calling them out.\n\nThe `smolfs`\n\ncommand is built from the Rust CLI crate. The GitHub workflow at\n`.github/workflows/publish-cli.yml`\n\nbuilds Linux and macOS release binaries for\nx86_64 and arm64 targets, smoke-tests `smolfs --help`\n\n, and attaches tarballs to\n`v*`\n\nGitHub releases. Pushes to `main`\n\npublish a rolling `dev`\n\nprerelease after\ndeleting the previous `dev`\n\nrelease and tag, so the dev channel only exposes the\nlatest successful main build.\n\nPython packaging uses `uv`\n\nand `maturin`\n\n. The GitHub workflow at\n`.github/workflows/publish-python.yml`\n\nbuilds wheels for Linux and macOS, builds\nan sdist, and publishes to PyPI when a `v*`\n\ntag is pushed.\n\nNode.js packaging uses `npm`\n\nand `napi-rs`\n\n. The GitHub workflow at\n`.github/workflows/publish-node.yml`\n\nbuilds native TypeScript SDK bindings for\nLinux and macOS, checks the npm package contents, and publishes to npm when a\n`v*`\n\ntag is pushed.\n\nBefore the first Python release, configure PyPI Trusted Publishing:\n\n- Create or claim the\n`smolfs`\n\nproject on PyPI. - Add a trusted publisher for repository\n`CelestoAI/smolfs`\n\n. - Set the workflow name to\n`publish-python.yml`\n\n. - Set the environment name to\n`pypi`\n\n.\n\nBefore the first npm release, configure npm trusted publishing for\n`@celestoai/smolfs`\n\n:\n\n- Add a trusted publisher for GitHub Actions.\n- Set the organization or user to\n`CelestoAI`\n\n. - Set the repository to\n`smolfs`\n\n. - Set the workflow filename to\n`publish-node.yml`\n\n. - Set the environment name to\n`npm`\n\n. - Allow\n`npm publish`\n\n.\n\nRelease:\n\n```\ngit tag v0.1.0\ngit push origin v0.1.0\n```\n\n- Add type stubs for the Python package.\n- Add a Linux CI job that mounts a local dev volume when mount support is available.\n- Add release notes and a changelog before the first non-draft release.\n\nApache 2.0.\n\n[Celesto AI](https://celesto.ai)", "url": "https://wpnews.pro/news/show-hn-a-durable-filesystem-layer-for-ai-agents", "canonical_source": "https://github.com/CelestoAI/smolfs", "published_at": "2026-06-25 00:05:53+00:00", "updated_at": "2026-06-25 00:13:32.138821+00:00", "lang": "en", "topics": ["ai-agents", "developer-tools", "ai-infrastructure"], "entities": ["CelestoAI", "SmolFS", "Redis", "S3", "Python", "TypeScript", "Rust", "SQLite"], "alternates": {"html": "https://wpnews.pro/news/show-hn-a-durable-filesystem-layer-for-ai-agents", "markdown": "https://wpnews.pro/news/show-hn-a-durable-filesystem-layer-for-ai-agents.md", "text": "https://wpnews.pro/news/show-hn-a-durable-filesystem-layer-for-ai-agents.txt", "jsonld": "https://wpnews.pro/news/show-hn-a-durable-filesystem-layer-for-ai-agents.jsonld"}}