{"slug": "caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models", "title": "Caddy – an open source modular viewer for 3D Gaussian Splatting models", "summary": "Emir Yüksel released Caddy, an open-source modular viewer for 3D Gaussian Splatting models that supports linking multiple PLY checkpoints, custom renderers, and remote GPU rendering. The tool allows users to visualize results locally via a session graph and requires an NVIDIA GPU with CUDA for local rendering, while remote clients can stream frames without a GPU.", "body_md": "*Modular viewer for 3D Gaussian Splatting models.*\n\nLink multiple PLY checkpoints, plug in custom renderers, render on remote GPU servers, and visualize results locally — with a session graph to jump between linked models.\n\nEmir Yüksel ·\nemir_yuksel@yahoo.com ·\nLinkedIn\n\n[1. Prerequisites](#1-prerequisites)[2. Get the code](#2-get-the-code)[3. Create the environment](#3-create-the-environment)[Viewing results in Caddy](#viewing-results-in-caddy)[Custom renderers (](#custom-renderers-user_renderers)`user_renderers/`\n\n)[Rendering / debugging helpers](#rendering--debugging-helpers)[Shared modules (not entry points)](#shared-modules-not-entry-points)\n\nThese are **not** installed by the conda environment and must exist beforehand:\n\n**NVIDIA GPU + CUDA**— required only on the machine that** renders**. For** local**rendering that is your own machine. For** remote**rendering the GPU is on the server, so the** client machine needs no GPU/CUDA**— it only displays frames streamed from the server.** A 3DGS trainer**(e.g.[graphdeco-inria gaussian-splatting](https://github.com/graphdeco-inria/gaussian-splatting)) — to train Gaussians before viewing them in Caddy. Caddy only*views*the result.**OpenSSH client**— only if you use** remote**GPU rendering.\n\nRemote viewing requires Caddy on the server too.The remote GPU server must have this repository checked out (at the path you set asremote project pathinSettings) with a working conda env (default name`caddy`\n\n). OnConnect to server, the client starts`scripts/RemoteVisualization/visualize_ply_ws_server.py`\n\nover SSH inside that env, then streams rendered frames back. If the repo or env is missing on the server, the connection will fail.\n\n```\ngit clone https://github.com/emiryuksel02/Caddy.git Caddy\ncd Caddy\nconda env create -f requirements.yaml\nconda activate caddy\n\n# CUDA rasterizer — only needed for LOCAL GPU rendering (needs a CUDA toolchain / nvcc).\n# Skip this entirely for remote-only viewing; the server does the rendering.\npip install git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git\n```\n\nNotes:\n\n-\n**Remote-only client?** You can skip the`diff-gaussian-rasterization`\n\ninstall above. Caddy imports it lazily, so the viewer launches and connects to a remote server without it. You'll only get`No module named 'diff_gaussian_rasterization'`\n\nif you try to render**locally** without it installed. -\nOn a CUDA machine, install the CUDA build of PyTorch (e.g. add the\n\n`nvidia`\n\nchannel and a`pytorch-cuda=<version>`\n\npin) instead of the default CPU build. The bundled`requirements.yaml`\n\nalready pins`pytorch-cuda=12.1`\n\n; adjust if your driver/CUDA stack differs.\n\nQuick smoke test (needs a PLY and a CUDA GPU):\n\n```\npython scripts/render_one_frame.py --ply point_cloud.ply --out test.png\n```\n\nHeadless single-frame render to PNG (smoke test for the CUDA renderer).\n\n`scripts/render_one_frame.py`\n\n```\npython scripts/render_one_frame.py --ply \"point_cloud.ply\" --out \"frame.png\"\n```\n\nTest a custom plugin renderer (see [Custom renderers](#custom-renderers-user_renderers)):\n\n```\npython scripts/render_one_frame.py `\n  --ply \"point_cloud.ply\" `\n  --renderer user_envmap `\n  --out \"frame_envmap.png\"\n```\n\n**Optional parameters:**\n\n`--renderer`\n\n— renderer name: built‑ins (default`cuda_diff_gaussian`\n\n) plus any plugin registered from`user_renderers/`\n\n.`--width`\n\n/`--height`\n\n— output resolution (default:`1024`\n\n×`768`\n\n).`--distance`\n\n— initial camera distance (default:`3.0`\n\n).\n\nLoad a `.ply`\n\nvia drag-and-drop or Browse. For remote GPU rendering, open **Settings**,\nenter your server profile (SSH target, port, remote Caddy path, conda env), click\n**Save**, then **Connect to server** (SSH tunnel and render server are started\nautomatically).\n\n`scripts/start_Caddy.py`\n\n```\npython -m scripts.start_Caddy\n```\n\nRunning on WSL/Linux?The Qt window needs a few system libraries that are not pulled in by conda/pip. If you see`Could not load the Qt platform plugin \"xcb\"`\n\nor`libxcb-*`\n\nerrors, install them once:\n\n```\nsudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1\nexport QT_QPA_PLATFORM=xcb   # only if the platform plugin still isn't picked up\n```\n\nIn the viewer: use the **Settings** renderer dropdown to switch renderers.\nRight‑click a model in the session graph → **Set renderer for this model…** to\npick a different renderer per linked PLY (local and remote modes).\n\nAll movement is **camera-local** (first-person style): arrow keys strafe and\nmove forward/back in the camera frame; rotations are around the camera’s own\naxes. **Local** mode uses the same key map as the remote client (the same\nwire-protocol strings the GPU server expects).\n\nFocus the main viewport (click the image) before using keyboard shortcuts.\n\n| Input | Action |\n|---|---|\n↑ / ↓ |\nMove forward / backward (along view axis) |\n← / → |\nStrafe left / right |\nShift + ↑ |\nAscend along camera-local +Y |\nShift + ↓ |\nDescend along camera-local −Y |\nW / S |\nPitch up / down |\nA / D |\nYaw left / right |\nQ / E |\nRoll counter-clockwise / clockwise |\n+ / − (or = / −) |\nZoom out / in |\nCtrl + G |\nOpen Go to dialog — teleport camera to a world (x, y, z) |\nStatus bar X Y Z |\nEdit camera position inline (Enter or focus-out applies) |\n\n| Input | Action |\n|---|---|\nScroll wheel |\nMove forward / backward (same as + / −) |\nClick colored link marker in the viewport |\nJump to the linked model (loads target PLY) |\n| Hover link marker | Cursor changes to a hand when a link is under the pointer |\n\n| Input | Action |\n|---|---|\nDrag & drop `.ply` onto the window |\nLoad locally or upload to remote server |\nBrowse / Load another… |\nPick a PLY from disk |\nSettings → Connect to server |\nSSH tunnel + remote WebSocket render server |\n\nOpen **Session** (bottom-right of the status bar) to view models as nodes and\nlinks as directed edges.\n\nPress **L** at a camera pose to create a link to another model. Configure the\nmarker color, label, and target renderer in the dialog; the sphere appears in\nthe viewport and jumps to the linked PLY when clicked.\n\n| Input | Action |\n|---|---|\nLeft-click a node |\nLoad that model and close the dialog |\nRight-click a node |\nSet renderer for this model…, Remove a link…, or Remove node from graph |\nRight-click an edge |\nRemove link |\nL |\nCreate a link at the current camera position to another model (pick target + marker style in the dialog) |\nClick × on an edge (after Remove a link…) |\nDelete that link |\nLoad session… / Save session… |\nRead/write a `.caddy` session file (models + links + per-model renderers) |\n\n| Input | Action |\n|---|---|\nSettings → Renderer |\nChoose the active renderer (local or after remote connect) |\n1 – 9, 0 |\nJump to saved camera preset from `camera_presets.json` (if present) |\n\nCaddy supports **plugin renderers** for experimenting with alternate shading\npipelines (e.g. env‑map or GaussianShader‑style models) on top of the same 3DGS\ncore. Plugins are auto‑discovered at startup from a `user_renderers/`\n\nfolder at\nthe repository root (create it if missing).\n\nA full working example lives in ** user_renderers/render.py** (registers the\n\n`user_envmap`\n\nrenderer with the adapter/wrapper pattern below).Add a Python file, e.g. `user_renderers/my_renderer.py`\n\n:\n\n``` python\nfrom typing import Dict\n\nimport torch\n\nfrom gaussian_splatting.gaussian_renderer import I3DGSRasterizer\nfrom gaussian_splatting.scene.scene_interface import I3DGSScene\n\nclass MyRenderer(I3DGSRasterizer):\n    def __init__(self) -> None:\n        self._scene: I3DGSScene | None = None\n\n    # Optional: build scene from PLY (called once per load)\n    def load_scene(self, ply_path: str) -> I3DGSScene:\n        raise NotImplementedError\n\n    # Required: one frame\n    def render(\n        self,\n        viewpoint_camera,\n        scene: I3DGSScene,\n        bg_color: torch.Tensor,\n        scaling_modifier: float = 1.0,\n        override_color: torch.Tensor | None = None,\n        debug: bool = False,\n    ) -> dict[str, torch.Tensor]:\n        raise NotImplementedError\n\ndef get_renderers() -> Dict[str, callable]:\n    return {\"my_renderer\": lambda: MyRenderer()}\n```\n\n**Registration:** on startup Caddy scans `user_renderers/`\n\n, imports each `.py`\n\n,\nand calls `get_renderers()`\n\n. Names are added to\n`gaussian_splatting.renderer_registry`\n\n.\n\n** I3DGSScene** exposes at minimum\n\n`means3D`\n\n, `scales`\n\n, `rotations`\n\n, `opacity`\n\n;\noptional `colors`\n\n, `features`\n\n, and `extra_attrs`\n\nfor custom per‑splat data.The built‑in default renderer ** cuda_diff_gaussian** is itself an adapter:\n\n`DiffGaussianRasterizerAdapter`\n\nimplements `I3DGSRasterizer`\n\nand renders any\n`I3DGSScene`\n\nvia `diff_gaussian_rasterization`\n\n. See\n`gaussian_splatting/examples/example_scenes.py`\n\nfor a minimal usage example.If you already have a render function from another 3DGS project (e.g.\nGaussianShader) that expects a model object `pc`\n\nwith `get_xyz`\n\n, `get_features`\n\n,\nBRDF fields, etc., you usually need **two wrappers**:\n\n**Scene wrapper (**— on`…Scene(I3DGSScene)`\n\n)**load**, wrap your native model so Caddy can hold and pass it as a generic`I3DGSScene`\n\n.**Reverse adapter (**— on`_SceneAdapter`\n\n)**render**, wrap the incoming`I3DGSScene`\n\nback into the API your legacy`render()`\n\nexpects.\n\n```\nload_scene(ply)\n  → GaussianShaderScene(GaussianModel)     # native model → I3DGSScene\n\nUserEnvMapRenderer.render(scene)\n  → pc = _SceneAdapter(scene)              # I3DGSScene → pc-shaped API\n  → render(viewpoint_camera, pc, pipe, …)  # your existing code unchanged\n```\n\n**Load path** (`user_renderers/render.py`\n\n): `UserEnvMapRenderer.load_scene`\n\nloads a GaussianShader‑style checkpoint (PLY + optional `brdf_mlp/…/brdf_mlp.hdr`\n\n)\nand returns a `GaussianShaderScene`\n\nthat maps `get_xyz`\n\n→ `means3D`\n\n, etc.\n\n**Render path:** `UserEnvMapRenderer.render`\n\ncalls `_SceneAdapter(scene)`\n\nso the\ngeneric `I3DGSScene`\n\nfrom Caddy looks like a `GaussianModel`\n\nagain, then delegates\nto the module’s existing `render()`\n\nfunction. A small `_ShaderPipe`\n\nobject stands\nin for GaussianShader’s pipeline/config.\n\nTest the bundled example:\n\n```\npython scripts/render_one_frame.py `\n  --ply \"path/to/point_cloud.ply\" `\n  --renderer user_envmap `\n  --out \"frame_envmap.png\"\n```\n\nYou do **not** have to rewrite your shading code — only implement the two\ninterface classes (`I3DGSScene`\n\nwrapper + `I3DGSRasterizer`\n\n) and a thin adapter\nbetween them and your existing `render()`\n\n.\n\nMinimal renderer test with a camera at the origin. Use to check the rendering environment.\n\n`scripts/render_single_ply.py`\n\n```\npython scripts/render_single_ply.py `\n  --ply \"point_cloud.ply\" `\n  --out \"test.png\"\n```\n\n**Required parameters:**\n\n`--ply`\n\n**Optional parameters:**\n\n`--width`\n\n`--height`\n\n`--out`\n\n`scripts/caddy_core.py`\n\n— rendering backends (`LocalRenderer`\n\n,`RemoteRenderer`\n\n) used by the viewer and headless render scripts.`scripts/camera_math.py`\n\n— camera/rotation math helpers.\n\n— this package is only needed for`No module named 'diff_gaussian_rasterization'`\n\n**local** GPU rendering. For remote-only viewing you can ignore it; to render locally, install it (see[Section 3](#3-create-the-environment)).— install the Qt system libs:`Could not load the Qt platform plugin \"xcb\"`\n\n/`libxcb-*`\n\nerrors (WSL/Linux)`sudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1`\n\n, and if needed`export QT_QPA_PLATFORM=xcb`\n\n. Run`ldd <PySide6>/Qt/plugins/platforms/libqxcb.so | grep \"not found\"`\n\nto see any remaining missing library.— make sure the repo and conda env exist on the`Failed to start remote server`\n\n/ connection closed**server** at the configured*remote project path*, and that the SSH target, port, and conda env name in**Settings** are correct.— set`OMP: Error #15 ... libiomp5md.dll already initialized`\n\n(Windows)`KMP_DUPLICATE_LIB_OK=TRUE`\n\nbefore launching (e.g. PowerShell:`$env:KMP_DUPLICATE_LIB_OK=\"TRUE\"`\n\n).— use the pinned`DLL load failed while importing QtCore`\n\n(Windows)`PySide6==6.7.3`\n\nfrom`requirements.yaml`\n\n; newer PySide6 builds can require a newer MSVC runtime than conda ships.", "url": "https://wpnews.pro/news/caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models", "canonical_source": "https://github.com/emiryuksel02/Caddy", "published_at": "2026-06-25 17:27:27+00:00", "updated_at": "2026-06-25 17:44:02.856713+00:00", "lang": "en", "topics": ["computer-vision", "ai-tools"], "entities": ["Emir Yüksel", "Caddy", "NVIDIA", "CUDA", "OpenSSH", "graphdeco-inria"], "alternates": {"html": "https://wpnews.pro/news/caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models", "markdown": "https://wpnews.pro/news/caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models.md", "text": "https://wpnews.pro/news/caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models.txt", "jsonld": "https://wpnews.pro/news/caddy-an-open-source-modular-viewer-for-3d-gaussian-splatting-models.jsonld"}}