{"slug": "private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata", "title": "Private User Document Storage for PDF, DOCX, Invoices, and Tenant Metadata", "summary": "A developer outlined a pattern for storing user-uploaded documents such as PDFs, DOCX files and invoices in private object storage while making a tenant-scoped relational metadata record the sole authority for discovering and delivering them. The approach derives tenant IDs from server-side authentication, generates opaque document IDs, keeps original filenames as display-only data, and queries by both document ID and tenant ID before fetching bytes rather than checking ownership after retrieval. The writeup also recommends versioning parsed text, chunks and embeddings against source checksums so AI pipelines can attribute costs and avoid re-processing identical bytes.", "body_md": "Short answer: store uploaded bytes in private object storage, but make a tenant-scoped database record the only way an application can discover or deliver them. Prefixes organize keys; they do not authorize a download.\n\nFor a product handling user uploads, the decisive question is not whether the object is a PDF, DOCX, or invoice. It is where the system makes its access decision. Put that decision in a query constrained by the authenticated tenant, keep generated identifiers separate from user-supplied names, and treat the object store as a byte service. That gives a Python service a small control plane to test before a document reaches a parser, an embedding job, or a model prompt.\n\nTrust the query.\n\nUse one logical document row for each upload. It should carry the tenant ID, uploader, opaque document ID, original filename, claimed media type, byte count, checksum, object key, and lifecycle state. The original name is display data. It is never a canonical key segment or an authorization input.\n\nThe object key can still be legible enough for operations: `tenants/{tenant_id}/documents/{document_id}/v1`. A prefix helps inventory, targeted retention work, and incident investigation. It does not stop a caller who has somehow obtained a key. Private storage plus a scoped metadata lookup does.\n\nFor an AI workflow, preserve the distinction between source bytes and derivatives. Parsed text, chunks, embeddings, and generated summaries each need a reference to a specific checksum or document version. Otherwise a retry can quietly parse and tokenize identical bytes again, which makes cost attribution and evaluation results fuzzy.\n\nKeep the states few: `pending`, `ready`, `quarantined`, and `deleted` are usually enough. Readers see only `ready`; a deletion request makes a record ineligible before cleanup work removes the bytes. Small rules, large payoff.\n\n| Concern | Authoritative location | Why it matters | \n|---|---|---|\n| Object bytes | Private object storage | It keeps large payloads out of ordinary database reads and backups. | \n| Tenant ownership and state | Relational metadata | A scoped query can decide access before any object key is used. | \n| Original filename | Metadata | It is presentation data, not a storage address. | \n\nThe service should derive `tenant_id` from server-side authentication and generate `document_id` itself. It must not accept either value from the browser as a trusted storage location. On upload, write the object, then create the metadata row; if the metadata transaction cannot commit, remove the object. On download, query by both document ID and tenant ID before requesting bytes. Do not fetch by ID and check ownership later.\n\nHere is a compact Python version. The in-memory object store keeps the important ordering and authorization behavior visible without tying the article to a provider.\n\n``` python\nfrom __future__ import annotations\n\nimport hashlib\nimport sqlite3\nimport uuid\nfrom dataclasses import dataclass\n\n@dataclass(frozen=True)\nclass AuthContext:\n    tenant_id: str\n    user_id: str\n\nclass PrivateObjectStore:\n    def __init__(self) -> None:\n        self._objects: dict[str, bytes] = {}\n\n    def put(self, key: str, body: bytes) -> None:\n        self._objects[key] = body\n\n    def delete(self, key: str) -> None:\n        self._objects.pop(key, None)\n\ndef create_database() -> sqlite3.Connection:\n    db = sqlite3.connect(\":memory:\")\n    db.execute(\n        \"\"\"\n        CREATE TABLE document_metadata (\n            id TEXT PRIMARY KEY,\n            tenant_id TEXT NOT NULL,\n            uploaded_by TEXT NOT NULL,\n            original_name TEXT NOT NULL,\n            byte_count INTEGER NOT NULL,\n            sha256 TEXT NOT NULL,\n            object_key TEXT NOT NULL UNIQUE,\n            state TEXT NOT NULL CHECK (state IN ('pending', 'ready', 'deleted'))\n        )\n        \"\"\"\n    )\n    return db\n\ndef save_upload(\n    auth: AuthContext,\n    original_name: str,\n    body: bytes,\n    objects: PrivateObjectStore,\n    db: sqlite3.Connection,\n) -> str:\n    if not body:\n        raise ValueError(\"an upload must contain bytes\")\n\n    document_id = str(uuid.uuid4())\n    key = f\"tenants/{auth.tenant_id}/documents/{document_id}/v1\"\n    digest = hashlib.sha256(body).hexdigest()\n    objects.put(key, body)\n    try:\n        with db:\n            db.execute(\n                \"\"\"\n                INSERT INTO document_metadata\n                    (id, tenant_id, uploaded_by, original_name, byte_count,\n                     sha256, object_key, state)\n                VALUES (?, ?, ?, ?, ?, ?, ?, 'ready')\n                \"\"\",\n                (document_id, auth.tenant_id, auth.user_id, original_name,\n                 len(body), digest, key),\n            )\n    except Exception:\n        objects.delete(key)\n        raise\n    return document_id\n\ndef locate_for_download(\n    auth: AuthContext, document_id: str, db: sqlite3.Connection\n) -> tuple[str, str] | None:\n    row = db.execute(\n        \"\"\"\n        SELECT original_name, object_key\n        FROM document_metadata\n        WHERE id = ? AND tenant_id = ? AND state = 'ready'\n        \"\"\",\n        (document_id, auth.tenant_id),\n    ).fetchone()\n    return None if row is None else (row[0], row[1])\n```\n\nThis is deliberately plain. The paired predicates are the important line: a cross-tenant ID has no matching metadata record, so the delivery path never receives an object key. A security test should create two tenants, upload under one, and assert that every public read operation returns no document for the other. Run it against list, download, preview, extraction, and deletion paths, because an overlooked secondary endpoint is where isolation tends to erode.\n\nAn object store and a relational database do not share one transaction. The code above handles the common failure window by compensating: delete a freshly written object when metadata insertion fails. For large or direct-to-storage uploads, make the gap explicit with a pending row and a finalize operation. A finalizer verifies the expected checksum and object key, then transitions the row to `ready` in the database transaction.\n\nThe failure sequence deserves more attention than the happy path. A client can finish sending bytes and disappear before finalization; a worker can retry after the browser has already shown success; a deletion can arrive while extraction is waiting; and a deployment can point a worker at the wrong environment. None of those cases should make an unscoped object discoverable. Record the intended tenant and document ID before accepting an asynchronous upload, require the finalizer to compare them with the expected key and checksum, and let only the metadata state expose the upload to a list or download handler. When a retry repeats, it should observe the same logical document rather than manufacture another one. This is also a useful place for an eval harness: exercise each state transition with a second tenant and confirm the transition never widens the query.\n\nThe catch is that this pattern is not suitable when a regulatory or contractual control requires a separately administered storage boundary for each tenant. In that case, use isolated storage accounts or buckets and accept the extra provisioning and policy work. Database binary columns can also be the better fit for very small, transaction-bound artifacts, although backups, replicas, and normal query traffic then carry the bytes too. There is no universal size cutoff; measure restore time, read volume, and operational ownership in the actual deployment.\n\nMake reconciliation a normal job, not an emergency script. It can identify aged pending rows, metadata whose objects are absent, and objects with no metadata owner. Cleanup and deletion must be idempotent, because queues retry. The same mindset applies to document ingestion: key parser and embedding work by the source checksum, then test that repeating an unchanged upload does not create a second derivative set.\n\nBrowser upload progress is useful feedback, but it is not proof that a completed object belongs to a user. The server still performs the authenticated metadata transition. Downloads need an equally deliberate response policy: private documents should start with conservative `Cache-Control` behavior, then receive integration tests for the browser and intermediary caches used by the application. Cache directives have different effects depending on the cache and response context, so validate the headers on real private download responses rather than copying a snippet from a public asset setup.\n\nDon't log document contents, raw authorization headers, or signed delivery material. Log a request ID, a tenant-safe internal identifier, document ID, lifecycle transition, byte count, and duration. Those fields can explain an upload path without turning observability into another document repository.\n\nBefore release, exercise a duplicate filename under two tenants, an attempted cross-tenant read, a failed metadata write after object creation, a repeated deletion, an old pending upload, and a restore into an isolated environment. Then run the extraction and retrieval evaluation twice against the same checksum. The second pass should be boring. Good.", "url": "https://wpnews.pro/news/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata", "canonical_source": "https://dev.to/jamesanderson121/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata-11af", "published_at": "2026-11-16 16:13:11+00:00", "updated_at": "2026-11-16 16:22:56.791398+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-agents"], "entities": [], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata", "markdown": "https://wpnews.pro/news/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata.md", "text": "https://wpnews.pro/news/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata.txt", "jsonld": "https://wpnews.pro/news/private-user-document-storage-for-pdf-docx-invoices-and-tenant-metadata.jsonld"}}