{"slug": "eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and", "title": "Eliminating TOCTOU Path Traversal in Python with Kernel-Enforced Containment and Lean 4 Formal Verification", "summary": "A developer built PathFence, a zero-dependency, pure-Python path containment library that replaces the standard realpath-then-open idiom with kernel-enforced confinement using openat2 with RESOLVE_BENEATH and RESOLVE_NO_SYMLINKS on Linux 5.6+, component-by-component openat with O_NOFOLLOW on macOS and older Linux, and reparse-point inspection on Windows. The library's decision function is formally verified in Lean 4, with theorems specifying exactly when a path is allowed, and a command-line tool exposes the proven function to any language. The author notes PathFence is v0.1.0, experimental, has had no human security review, and is not a sandbox because its decision is lexical.", "body_md": "When building AI agents, MCP servers, archive extractors, or multi-tenant web backends, confining file operations to a designated directory is critical.\n\nYet the standard Python idiom for path validation—normalizing with `os.path.realpath()` or `Path.resolve()`, checking a string prefix, and then calling `open()`—is fundamentally vulnerable to **TOCTOU** race conditions.\n\nTo solve this without requiring C extensions or sacrificing performance, I built **PathFence**: a zero-dependency, pure-Python path containment library backed by OS kernel primitives and mathematically verified in **Lean 4**.\n\nA path-confinement check for tools that let an AI agent (or any untrusted caller) read and write files, written and proven in Lean 4.\n\nYou give it the directories the caller may use, optionally directories inside them it may not use\nand a requested path. It answers *allow* (with the resolved path, and the allowed root it is under)\n*deny* (with the resolved path) or *reject* (with a reason). Lean theorems state exactly when it\nallows, and a command-line tool makes the proven function usable from any language.\n\n``` bash\n$ pathfence check --allow /tmp/allowed_dir -- /tmp/allowed_dir_malicious/secret ../x a/../b\n{\"input\":\"/tmp/allowed_dir_malicious/secret\",\"decision\":\"deny\",\"path\":\"/tmp/allowed_dir_malicious/secret\"}\n{\"input\":\"../x\",\"decision\":\"reject\",\"reason\":\"escapes\"}\n{\"input\":\"a/../b\",\"decision\":\"allow\",\"path\":\"/tmp/allowed_dir/b\",\"root\":\"/tmp/allowed_dir\",\"rel\":\"b\"}\n```\n\nStatus: v0.1.0, experimental. No human security review yet.\n\n**PathFence is not a sandbox. The decision is lexical.** It looks only at the path string, never\nat the file system. A symlink, a race between the check and the open, a hard link, a mount or a\ncase-insensitive file system…\n\nConsider the classic validation pattern found in thousands of production codebases:\n\n```\n# VULNERABLE: Classic TOCTOU race condition\nreal_root = os.path.realpath(root)\nreal_target = os.path.realpath(os.path.join(root, untrusted_path))\nif not real_target.startswith(real_root + os.sep):\n    raise PermissionError(\"Path traversal detected\")\n\n# An attacker can swap a path component with a symlink right here!\nwith open(real_target, \"rb\") as f:\n    data = f.read()\n```\n\nBetween the moment `os.path.realpath()` inspects the filesystem and the moment `open()` resolves the path again in the kernel, a local attacker or concurrent process can replace an intermediate directory with a symlink pointing to `/etc` or another sensitive location. Because the check and the use are two separate operations on path strings, the containment guarantee evaporates under concurrency.\n\n```\nsequenceDiagram\n    participant App as Python Process\n    participant Kernel as OS Kernel / VFS\n    participant Attacker as Concurrent Attacker\n\n    App->>Kernel: 1. realpath(\"/sandbox/work/data.txt\")\n    Kernel-->>App: \"/sandbox/work/data.txt\" - Prefix OK\n    Attacker->>Kernel: 2. rename & symlink \"/sandbox/work\" -> \"/etc\"\n    App->>Kernel: 3. open(\"/sandbox/work/passwd\")\n    Kernel-->>App: Returns file descriptor for /etc/passwd!\n```\n\n**PathFence** shifts the root of trust from mutable path strings to **pinned directory file descriptors**. Every operation resolves relative to an open directory handle using OS-level flags that forbid symlink traversal and upward escape at the kernel level.\n\n``` php\ngraph TD\n    Input[\"pathfence.safe_open / Workspace\"] --> Lex[\"Lexical Normalization & Windows Reserved Name Check\"]\n    Lex --> Platform{\"Platform Kernel Strategy\"}\n    Platform -->|Linux 5.6+| Openat2[\"openat2 syscall + RESOLVE_BENEATH / NO_SYMLINKS\"]\n    Platform -->|macOS / Older Linux| StepWalk[\"Component-by-component openat + O_NOFOLLOW + fstat\"]\n    Platform -->|Windows| WinHandle[\"Reparse Point Inspection + GetFinalPathNameByHandleW\"]\n    Openat2 --> Result[\"Verified File Descriptor / Stream\"]\n    StepWalk --> Result[\"Verified File Descriptor / Stream\"]\n    WinHandle --> Result[\"Verified File Descriptor / Stream\"]\n```\n\nOn modern Linux kernels, **PathFence** invokes the `openat2(2)` syscall directly via `ctypes` with four strict resolution flags:\n\n`RESOLVE_BENEATH`: Prevents resolution from escaping above the root directory file descriptor.`RESOLVE_NO_SYMLINKS`: Rejects all symbolic links anywhere in the path.` RESOLVE_NO_MAGICLINKS`: Blocks procfs magic-link traversal such as `/proc/self/fd/*`.` RESOLVE_NO_XDEV`: Prevents crossing mount-point boundaries.` O_NOFOLLOW` Walk\nOn macOS, BSD, and older Linux kernels, **PathFence** pins the root directory with `O_RDONLY | O_DIRECTORY | O_NOFOLLOW` and walks each path segment one at a time using `openat(..., O_NOFOLLOW)`. At each step, it verifies `fstat` device and inode identities to detect concurrent directory replacement.\n\nOn Windows, **PathFence** blocks reserved device names such as `CON`, `PRN`, `AUX`, `NUL`, `COM1`-` COM9`, and `LPT1`-` LPT9`, NTFS Alternate Data Streams containing `:`, trailing dots or spaces, and UNC/device prefixes. It inspects `FSCTL_GET_REPARSE_POINT` to reject junctions and mount points, and verifies the final handle path via `GetFinalPathNameByHandleW`.\n\nUnit tests and fuzzing can catch bugs, but they cannot prove the absence of edge cases across infinite path combinations. **PathFence** includes a complete formal specification written in **Lean 4** under the `formal/` directory.\n\nWe model path components, depth tracking, lexical normalization, and step-by-step directory resolution, proving **15 theorems with zero `sorry` placeholders and zero custom axioms**:\n\n| Theorem | Verified Security Property | \n|---|---|\n| `normalize_rejects_escape` | Any component sequence whose cumulative depth drops below zero is strictly rejected. | \n| `normalize_preserves_containment` | Every accepted path resolves strictly within the subtree rooted at the pinned base. | \n| `symlink_free_walk_stays_in_root` | Inductive proof that a symlink-free `O_NOFOLLOW` walk can never transition outside the root boundary. | \n| `normalize_idempotent` | Normalizing an already normalized path yields the exact same result. | \n| `beneath_composable` | Sub-workspaces created via `subworkspace()` transitively preserve parent containment guarantees. | \n\nBeyond opening files, `Workspace` confines the entire file lifecycle—including `mkdir`, `unlink`, `rmdir`, `rename`, `listdir`, and crash-safe `atomic_write`—inside the pinned boundary.\n\n``` python\nimport pathfence\nfrom pathfence import Workspace\n\n# One-liner safe open\nwith pathfence.safe_open(\"/var/sandbox\", \"notes/hello.txt\", \"r\") as f:\n    print(f.read())\n\n# Full lifecycle confinement for AI agents or file processors\nwith Workspace(\"/var/sandbox\") as ws:\n    ws.mkdir(\"output\", exist_ok=True)\n    ws.atomic_write(\"output/result.json\", b'{\"status\": \"verified\"}')\n\n    # Sub-workspaces are mathematically guaranteed to stay inside the parent\n    with ws.subworkspace(\"output\") as sub:\n        data = sub.read_bytes(\"result.json\")\n```\n\nBecause **PathFence** uses direct syscall bindings in pure Python, security comes with virtually zero latency penalty. Benchmarked over 100,000 iterations on Linux 6.8:\n\n`openat2` fast path`realpath` + `open` at 3.9 μs\n**PathFence** requires Python 3.9+ and has zero runtime dependencies.\n\n```\npip install pathfence\n```\n\nCheck out the repository, inspect the **Lean 4** proofs in `formal/`, and let me know your thoughts in the comments!\n\n``` bash\n$ pathfence check --allow /tmp/allowed_dir -- /tmp/allowed_dir_malicious/secret ../x a/../b\n{\"input\":\"/tmp/allowed_dir_malicious/secret\",\"decision\":\"deny\",\"path\":\"/tmp/allowed_dir_malicious/secret\"}\n{\"input\":\"../x\",\"decision\":\"reject\",\"reason\":\"escapes\"}\n{\"input\":\"a/../b\",\"decision\":\"allow\",\"path\":\"/tmp/allowed_dir/b\",\"root\":\"/tmp/allowed_dir\",\"rel\":\"b\"}\n```\n\n", "url": "https://wpnews.pro/news/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and", "canonical_source": "https://dev.to/genelab_999/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and-lean-4-formal-l64", "published_at": "2026-10-27 15:02:31+00:00", "updated_at": "2026-10-27 15:12:50.086536+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-safety"], "entities": ["PathFence", "Lean 4", "Python", "Linux", "macOS", "Windows", "openat2"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and", "markdown": "https://wpnews.pro/news/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and.md", "text": "https://wpnews.pro/news/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and.txt", "jsonld": "https://wpnews.pro/news/eliminating-toctou-path-traversal-in-python-with-kernel-enforced-containment-and.jsonld"}}