Caddy – an open source modular viewer for 3D Gaussian Splatting models

Emir Yüksel released Caddy, an open-source modular viewer for 3D Gaussian Splatting models that supports linking multiple PLY checkpoints, custom renderers, and remote GPU rendering. The tool allows users to visualize results locally via a session graph and requires an NVIDIA GPU with CUDA for local rendering, while remote clients can stream frames without a GPU.

Modular viewer for 3D Gaussian Splatting models. Link multiple PLY checkpoints, plug in custom renderers, render on remote GPU servers, and visualize results locally — with a session graph to jump between linked models. Emir Yüksel · emir yuksel@yahoo.com · LinkedIn 1. Prerequisites 1-prerequisites 2. Get the code 2-get-the-code 3. Create the environment 3-create-the-environment Viewing results in Caddy viewing-results-in-caddy Custom renderers custom-renderers-user renderers user renderers/ Rendering / debugging helpers rendering--debugging-helpers Shared modules not entry points shared-modules-not-entry-points These are not installed by the conda environment and must exist beforehand: NVIDIA GPU + CUDA — required only on the machine that renders . For local rendering that is your own machine. For remote rendering the GPU is on the server, so the client machine needs no GPU/CUDA — it only displays frames streamed from the server. A 3DGS trainer e.g. graphdeco-inria gaussian-splatting https://github.com/graphdeco-inria/gaussian-splatting — to train Gaussians before viewing them in Caddy. Caddy only views the result. OpenSSH client — only if you use remote GPU rendering. Remote viewing requires Caddy on the server too.The remote GPU server must have this repository checked out at the path you set asremote project pathinSettings with a working conda env default name caddy . OnConnect to server, the client starts scripts/RemoteVisualization/visualize ply ws server.py over SSH inside that env, then streams rendered frames back. If the repo or env is missing on the server, the connection will fail. git clone https://github.com/emiryuksel02/Caddy.git Caddy cd Caddy conda env create -f requirements.yaml conda activate caddy CUDA rasterizer — only needed for LOCAL GPU rendering needs a CUDA toolchain / nvcc . Skip this entirely for remote-only viewing; the server does the rendering. pip install git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git Notes: - Remote-only client? You can skip the diff-gaussian-rasterization install above. Caddy imports it lazily, so the viewer launches and connects to a remote server without it. You'll only get No module named 'diff gaussian rasterization' if you try to render locally without it installed. - On a CUDA machine, install the CUDA build of PyTorch e.g. add the nvidia channel and a pytorch-cuda=<version pin instead of the default CPU build. The bundled requirements.yaml already pins pytorch-cuda=12.1 ; adjust if your driver/CUDA stack differs. Quick smoke test needs a PLY and a CUDA GPU : python scripts/render one frame.py --ply point cloud.ply --out test.png Headless single-frame render to PNG smoke test for the CUDA renderer . scripts/render one frame.py python scripts/render one frame.py --ply "point cloud.ply" --out "frame.png" Test a custom plugin renderer see Custom renderers custom-renderers-user renderers : python scripts/render one frame.py --ply "point cloud.ply" --renderer user envmap --out "frame envmap.png" Optional parameters: --renderer — renderer name: built‑ins default cuda diff gaussian plus any plugin registered from user renderers/ . --width / --height — output resolution default: 1024 × 768 . --distance — initial camera distance default: 3.0 . Load a .ply via drag-and-drop or Browse. For remote GPU rendering, open Settings , enter your server profile SSH target, port, remote Caddy path, conda env , click Save , then Connect to server SSH tunnel and render server are started automatically . scripts/start Caddy.py python -m scripts.start Caddy Running on WSL/Linux?The Qt window needs a few system libraries that are not pulled in by conda/pip. If you see Could not load the Qt platform plugin "xcb" or libxcb- errors, install them once: sudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1 export QT QPA PLATFORM=xcb only if the platform plugin still isn't picked up In the viewer: use the Settings renderer dropdown to switch renderers. Right‑click a model in the session graph → Set renderer for this model… to pick a different renderer per linked PLY local and remote modes . All movement is camera-local first-person style : arrow keys strafe and move forward/back in the camera frame; rotations are around the camera’s own axes. Local mode uses the same key map as the remote client the same wire-protocol strings the GPU server expects . Focus the main viewport click the image before using keyboard shortcuts. | Input | Action | |---|---| ↑ / ↓ | Move forward / backward along view axis | ← / → | Strafe left / right | Shift + ↑ | Ascend along camera-local +Y | Shift + ↓ | Descend along camera-local −Y | W / S | Pitch up / down | A / D | Yaw left / right | Q / E | Roll counter-clockwise / clockwise | + / − or = / − | Zoom out / in | Ctrl + G | Open Go to dialog — teleport camera to a world x, y, z | Status bar X Y Z | Edit camera position inline Enter or focus-out applies | | Input | Action | |---|---| Scroll wheel | Move forward / backward same as + / − | Click colored link marker in the viewport | Jump to the linked model loads target PLY | | Hover link marker | Cursor changes to a hand when a link is under the pointer | | Input | Action | |---|---| Drag & drop .ply onto the window | Load locally or upload to remote server | Browse / Load another… | Pick a PLY from disk | Settings → Connect to server | SSH tunnel + remote WebSocket render server | Open Session bottom-right of the status bar to view models as nodes and links as directed edges. Press L at a camera pose to create a link to another model. Configure the marker color, label, and target renderer in the dialog; the sphere appears in the viewport and jumps to the linked PLY when clicked. | Input | Action | |---|---| Left-click a node | Load that model and close the dialog | Right-click a node | Set renderer for this model…, Remove a link…, or Remove node from graph | Right-click an edge | Remove link | L | Create a link at the current camera position to another model pick target + marker style in the dialog | Click × on an edge after Remove a link… | Delete that link | Load session… / Save session… | Read/write a .caddy session file models + links + per-model renderers | | Input | Action | |---|---| Settings → Renderer | Choose the active renderer local or after remote connect | 1 – 9, 0 | Jump to saved camera preset from camera presets.json if present | Caddy supports plugin renderers for experimenting with alternate shading pipelines e.g. env‑map or GaussianShader‑style models on top of the same 3DGS core. Plugins are auto‑discovered at startup from a user renderers/ folder at the repository root create it if missing . A full working example lives in user renderers/render.py registers the user envmap renderer with the adapter/wrapper pattern below .Add a Python file, e.g. user renderers/my renderer.py : python from typing import Dict import torch from gaussian splatting.gaussian renderer import I3DGSRasterizer from gaussian splatting.scene.scene interface import I3DGSScene class MyRenderer I3DGSRasterizer : def init self - None: self. scene: I3DGSScene | None = None Optional: build scene from PLY called once per load def load scene self, ply path: str - I3DGSScene: raise NotImplementedError Required: one frame def render self, viewpoint camera, scene: I3DGSScene, bg color: torch.Tensor, scaling modifier: float = 1.0, override color: torch.Tensor | None = None, debug: bool = False, - dict str, torch.Tensor : raise NotImplementedError def get renderers - Dict str, callable : return {"my renderer": lambda: MyRenderer } Registration: on startup Caddy scans user renderers/ , imports each .py , and calls get renderers . Names are added to gaussian splatting.renderer registry . I3DGSScene exposes at minimum means3D , scales , rotations , opacity ; optional colors , features , and extra attrs for custom per‑splat data.The built‑in default renderer cuda diff gaussian is itself an adapter: DiffGaussianRasterizerAdapter implements I3DGSRasterizer and renders any I3DGSScene via diff gaussian rasterization . See gaussian splatting/examples/example scenes.py for a minimal usage example.If you already have a render function from another 3DGS project e.g. GaussianShader that expects a model object pc with get xyz , get features , BRDF fields, etc., you usually need two wrappers : Scene wrapper — on …Scene I3DGSScene load , wrap your native model so Caddy can hold and pass it as a generic I3DGSScene . Reverse adapter — on SceneAdapter render , wrap the incoming I3DGSScene back into the API your legacy render expects. load scene ply → GaussianShaderScene GaussianModel native model → I3DGSScene UserEnvMapRenderer.render scene → pc = SceneAdapter scene I3DGSScene → pc-shaped API → render viewpoint camera, pc, pipe, … your existing code unchanged Load path user renderers/render.py : UserEnvMapRenderer.load scene loads a GaussianShader‑style checkpoint PLY + optional brdf mlp/…/brdf mlp.hdr and returns a GaussianShaderScene that maps get xyz → means3D , etc. Render path: UserEnvMapRenderer.render calls SceneAdapter scene so the generic I3DGSScene from Caddy looks like a GaussianModel again, then delegates to the module’s existing render function. A small ShaderPipe object stands in for GaussianShader’s pipeline/config. Test the bundled example: python scripts/render one frame.py --ply "path/to/point cloud.ply" --renderer user envmap --out "frame envmap.png" You do not have to rewrite your shading code — only implement the two interface classes I3DGSScene wrapper + I3DGSRasterizer and a thin adapter between them and your existing render . Minimal renderer test with a camera at the origin. Use to check the rendering environment. scripts/render single ply.py python scripts/render single ply.py --ply "point cloud.ply" --out "test.png" Required parameters: --ply Optional parameters: --width --height --out scripts/caddy core.py — rendering backends LocalRenderer , RemoteRenderer used by the viewer and headless render scripts. scripts/camera math.py — camera/rotation math helpers. — this package is only needed for No module named 'diff gaussian rasterization' local GPU rendering. For remote-only viewing you can ignore it; to render locally, install it see Section 3 3-create-the-environment .— install the Qt system libs: Could not load the Qt platform plugin "xcb" / libxcb- errors WSL/Linux sudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1 , and if needed export QT QPA PLATFORM=xcb . Run ldd <PySide6 /Qt/plugins/platforms/libqxcb.so | grep "not found" to see any remaining missing library.— make sure the repo and conda env exist on the Failed to start remote server / connection closed server at the configured remote project path , and that the SSH target, port, and conda env name in Settings are correct.— set OMP: Error 15 ... libiomp5md.dll already initialized Windows KMP DUPLICATE LIB OK=TRUE before launching e.g. PowerShell: $env:KMP DUPLICATE LIB OK="TRUE" .— use the pinned DLL load failed while importing QtCore Windows PySide6==6.7.3 from requirements.yaml ; newer PySide6 builds can require a newer MSVC runtime than conda ships.