I still see engineers storing user-uploaded photos inMeanwhile, the team next door threw the same photos into an S3 bucket and scaled to 100M files without breaking a sweat. The difference is the storage paradigm. Pick the wrong one and you feel it at scale./var/www/uploads/
on an ext4 volume and wondering why their server falls over at 10M files.
I still mix the two up in conversation sometimes, so I keep a short checklist: random writes and file locks mean file storage; HTTP PUTs and billions of objects mean object storage.
Short answer: use file storage when you need POSIX semantics — in-place edits, sub-millisecond random I/O, file locking (databases, OS files, NFS shares). Use object storage when you need scale, an HTTP API and rich metadata (user uploads, data lakes, backups, ML datasets). Most mature stacks run both, side by side.
| Fact | Value | Source |
|---|---|---|
| Amazon S3 consistency | Strong read-after-write for new objects, overwrites and LIST — since Dec 1, 2020, at no extra cost | |
| Dimension | File Storage |
Object Storage |
|---|---|---|
Data unit |
File (named byte sequence) | Object (data + metadata + key) |
Organization |
Hierarchical (directories/subdirectories) | Flat (key namespace; / is cosmetic) |
Access method |
POSIX (open/read/write/seek/close) | HTTP REST API (PUT/GET/DELETE) |
Mutability |
In-place (change bytes 100-200 without touching 1-99) | Immutable (overwrite = new version/new object) |
Metadata |
Fixed attributes (name, size, permissions, timestamps) | Rich & extensible (custom key-value tags, content-type, etc.) |
Scaling limit |
Millions of files (inode exhaustion, metadata perf) | Billions+ of objects (distributed metadata) |
Protocol |
NFS, SMB, POSIX local (ext4, xfs, zfs) | S3 API (HTTP/HTTPS) |
Consistency model |
Strong (reads see writes immediately) | Strong read-after-write on AWS S3 since Dec 2020 — covers new objects, overwrites and LIST; S3-compatible systems vary |
Typical latency |
Sub-millisecond (local) to milliseconds (NFS) | Milliseconds (network round-trip) |
Best for |
OS-level operations, databases, home dirs | Unstructured data at scale, web/mobile apps, analytics |
File storage is the right choice when your application (or OS) needs POSIX semantics:
/etc/hosts
/var/log/syslog
/home/user/.bashrc
/tmp/processing_12345.tmp
Your OS expects file storage. It uses open()
, read()
, write()
, seek()
— not HTTP PUT/GET. Don't fight this.
PostgreSQL, MySQL, MongoDB, SQLite — they all expect block devices or file systems with:
.lock
files, advisory locks)Object storage has millisecond-level latency and no in-place mutation. Databases on S3 perform terribly (with niche exceptions like Iceberg/Delta lakehouse patterns).
When multiple users/servers need shared access to the same files with familiar tools:
These use cases need file-level permissions, directory browsing, and application transparency — all strengths of file storage.
For small datasets, file storage is simpler:
ls
, cp
, rsync
, grep
)Object storage is the right choice when you need scale, simplicity of API, and rich metadata:
Photos, videos, documents, uploads — the canonical object storage workload:
s3.put_object(
Bucket="user-photos",
Key=f"user-{user_id}/photo-{uuid}.jpg",
Body=image_data,
ContentType="image/jpeg",
Metadata={"up": str(user_id), "camera": "iphone"}
)
Scale from 1K to 100M objects without changing code. I have watched teams try to stretch a filesystem to that size; inode exhaustion is not a fun afternoon.
Parquet/Avro/CSV files for Spark, Trino, DuckDB:
s3://data-lake/bronze/events/year=2026/month=07/day=24/event-*.parquet
s3://data-lake/gold/daily_active_users.parquet
Flat namespace, massive scale, accessed by query engines that speak S3 natively. This is where object storage dominates in 2026.
Database dumps, VM snapshots, compliance records:
File storage can do backups too, but at scale, object storage's tiering and replication features save significant cost and operational effort.
S3 + CloudFront (or Cloudflare) is the standard pattern for serving static web content:
Serving a global static site from an NFS mount is not something I'd want to run on-call for.
Training data, model checkpoints, inference outputs:
ML workloads at scale (terabytes of training data) are almost always object-storage-backed in 2026.
What if you want S3's scale but need file-system semantics? I have been asked this in almost every object-storage migration. The honest answer is: you can, but the mount layer will lie to you in small ways. Here is what each project officially documents:
| Tool | Language | What it does | Officially documented limits |
|---|---|---|---|
| C++ | Mounts an S3 bucket via FUSE on Linux/macOS/FreeBSD; preserves the native object format so aws s3 still works |
||
| "random writes or appends to files require rewriting the entire object"; "no atomic renames of files or directories"; "no hard links"; "no coordination between multiple clients mounting the same bucket" | |||
| Go | A "Filey System" that "strives for performance first and POSIX second"; close-to-open consistency, no on-disk cache | "only sequential writes supported"; "does not support symlink or hardlink"; "cannot rename directories with more than 1000 children"; "fsync is ignored" — and the last commit was June 2023, so treat it as low-maintenance |
|
| Rust | AWS's own GA file client, tuned for high read throughput and sequential writes of new objects | AWS states it is "probably not the right fit" for apps that use "directory renaming or symlinks" or "make edits to existing files (don't work on your Git repository or run vim in Mountpoint)"; support for non-AWS S3-compatible stores is limited |
|
| Go | Mounts any of rclone's 70+ backends, including any S3-compatible endpoint | rclone's own docs warn the file system is not fully POSIX-compliant; behaviour depends on VFS cache mode |
Own it first: RustFS does not ship a FUSE driver. Its README Feature & Status table covers S3 core, versioning, bucket replication, event notifications, bitrot protection, Swift/Keystone and Helm charts — no POSIX mount. If you want a mount, point one of the clients above at RustFS's S3 endpoint like you would at any other S3 service. Anyone telling you a "native RustFS mount" exists is reading a spec sheet that doesn't.
Performance reality: the translation layer is POSIX → HTTP, so each metadata operation becomes a network round trip. s3fs-fuse names this explicitly: "metadata operations such as listing directories have poor performance due to network latency." That's fine for bulk work — cp
, tar
, grep
, feeding a training job. It is not fine for databases, build systems or anything doing high-IOPS random writes, because those turn into whole-object rewrites.
Do you need sub-millisecond random I/O?
├─ YES → File Storage (database, OS files)
│ (or block storage)
│
└─ NO → Do you need POSIX semantics (ls, chmod, flock)?
├─ YES → File Storage (NFS/SMB shares, source code)
│
└─ NO → Will you exceed 1M files/objects?
├─ YES → Object Storage (S3/S3-compatible)
│ (photos, data lake, backups, ML)
│
└─ NO → Either works; pick the simpler tool
for your team's skill set
rclone mount
)Need S3-compatible object storage you can run yourself? RustFS is Apache 2.0 licensed (no AGPL strings) and its README lists S3 core, versioning, bucket replication, event notifications, bitrot protection, multi-tenancy and Helm charts as Available; Lifecycle Management, Distributed Mode and KMS are still marked Under Testing — so plan accordingly. Try it in one command:
docker run -d -p 9000:9000 -p 9001:9001 -v $(pwd)/data:/data -v $(pwd)/logs:/logs rustfs/rustfs:latest
[sourced verbatim from the RustFS GitHub README — NOT EXECUTED IN CI]. Console on port 9001, default credentials rustfsadmin / rustfsadmin — change them before you expose anything. Binaries and the rc CLI: rustfs.com/download.
Sometimes. For plain file sharing I usually skip the FUSE shim and serve objects through a web UI or pre-signed URLs — one less POSIX lie to debug. If you really need a mount, read the limits first: s3fs-fuse has "no atomic renames of files or directories" and "no coordination between multiple clients mounting the same bucket"; goofys supports "only sequential writes"; Mountpoint refuses edits to existing files. Compilers, build systems, anything calling flock()
— those stay on real file storage.
It depends where the reader is sitting. A local NVMe filesystem wins for a single machine; object storage wins when you need a CDN in front of it. The question I ask is not "which is faster" but "which is fast enough at this distance". Databases need the local path. A photo served worldwide needs the CDN path. I ignore quoted millisecond figures unless they come with the test setup attached.
Traditional OLTP databases — PostgreSQL, MySQL — no, not as their primary data directory. They need in-place mutation, ordered fsync
and sub-millisecond random reads, none of which object storage provides. What does work, and works extremely well, is the lakehouse pattern: query engines such as DuckDB (via httpfs
), Trino, Spark and ClickHouse (S3 table engine) read Parquet/ORC directly out of S3, and table formats like Apache Iceberg and Delta Lake add ACID semantics on top of immutable objects. Object storage is also the universal backup target for databases. So the accurate statement is: analytics on object storage, yes; transactional storage engine on object storage, no.
Gradually, and by workload rather than by directory. A path that works: (1) point all new workloads at S3 from day one; (2) move user-generated content first — uploads are the natural fit; (3) move analytics data next, as Parquet in a bucket queried by Spark/Trino/DuckDB; (4) leave the legacy file server on NFS/SMB and mirror it to object storage for DR and archive; (5) never move OS files. rclone sync
handles filesystem-to-S3 copies against any S3-compatible endpoint, and aws s3 sync
works for AWS. Budget for a coexistence period — both paradigms running side by side is the normal end state, not a failure.
No. As of the check date on this article, the RustFS GitHub README's Feature & Status table lists S3 Core Features, Upload/Download, Versioning, Logging, Event Notifications, K8s Helm Charts, Keystone Auth, Swift API, Bitrot Protection, Single Node Mode, Bucket Replication and Multi-Tenancy as Available, with Lifecycle Management, Distributed Mode and RustFS KMS marked Under Testing. There is no FUSE driver, no rustfs mount
command and no POSIX mount feature anywhere in the README or on docs.rustfs.com. If you need a mount, run s3fs-fuse, goofys or rclone mount
against the RustFS S3 endpoint on port 9000 — exactly as you would against any other S3-compatible service.
All claims above were checked against primary sources on 2026-08-07:
fsync
ignored); last commit June 2023 — rclone mount
documentation —