Modular viewer for 3D Gaussian Splatting models.
Link multiple PLY checkpoints, plug in custom renderers, render on remote GPU servers, and visualize results locally — with a session graph to jump between linked models.
Emir Yüksel · emir_yuksel@yahoo.com · LinkedIn
1. Prerequisites2. Get the code3. Create the environmentViewing results in CaddyCustom renderers (user_renderers/
)Rendering / debugging helpersShared modules (not entry points)
These are not installed by the conda environment and must exist beforehand:
NVIDIA GPU + CUDA— required only on the machine that** renders**. For** localrendering that is your own machine. For remoterendering the GPU is on the server, so the client machine needs no GPU/CUDA**— it only displays frames streamed from the server.** A 3DGS trainer**(e.g.graphdeco-inria gaussian-splatting) — to train Gaussians before viewing them in Caddy. Caddy onlyviewsthe result.OpenSSH client— only if you use** remote**GPU rendering.
Remote viewing requires Caddy on the server too.The remote GPU server must have this repository checked out (at the path you set asremote project pathinSettings) with a working conda env (default namecaddy
). OnConnect to server, the client startsscripts/RemoteVisualization/visualize_ply_ws_server.py
over SSH inside that env, then streams rendered frames back. If the repo or env is missing on the server, the connection will fail.
git clone https://github.com/emiryuksel02/Caddy.git Caddy
cd Caddy
conda env create -f requirements.yaml
conda activate caddy
pip install git+https://github.com/graphdeco-inria/diff-gaussian-rasterization.git
Notes:
Remote-only client? You can skip thediff-gaussian-rasterization
install above. Caddy imports it lazily, so the viewer launches and connects to a remote server without it. You'll only getNo module named 'diff_gaussian_rasterization'
if you try to renderlocally without it installed. - On a CUDA machine, install the CUDA build of PyTorch (e.g. add the
nvidia
channel and apytorch-cuda=<version>
pin) instead of the default CPU build. The bundledrequirements.yaml
already pinspytorch-cuda=12.1
; adjust if your driver/CUDA stack differs.
Quick smoke test (needs a PLY and a CUDA GPU):
python scripts/render_one_frame.py --ply point_cloud.ply --out test.png
Headless single-frame render to PNG (smoke test for the CUDA renderer).
scripts/render_one_frame.py
python scripts/render_one_frame.py --ply "point_cloud.ply" --out "frame.png"
Test a custom plugin renderer (see Custom renderers):
python scripts/render_one_frame.py `
--ply "point_cloud.ply" `
--renderer user_envmap `
--out "frame_envmap.png"
Optional parameters:
--renderer
— renderer name: built‑ins (defaultcuda_diff_gaussian
) plus any plugin registered fromuser_renderers/
.--width
/--height
— output resolution (default:1024
×768
).--distance
— initial camera distance (default:3.0
).
Load a .ply
via drag-and-drop or Browse. For remote GPU rendering, open Settings, enter your server profile (SSH target, port, remote Caddy path, conda env), click Save, then Connect to server (SSH tunnel and render server are started automatically).
scripts/start_Caddy.py
python -m scripts.start_Caddy
Running on WSL/Linux?The Qt window needs a few system libraries that are not pulled in by conda/pip. If you seeCould not load the Qt platform plugin "xcb"
orlibxcb-*
errors, install them once:
sudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1
export QT_QPA_PLATFORM=xcb # only if the platform plugin still isn't picked up
In the viewer: use the Settings renderer dropdown to switch renderers. Right‑click a model in the session graph → Set renderer for this model… to pick a different renderer per linked PLY (local and remote modes).
All movement is camera-local (first-person style): arrow keys strafe and move forward/back in the camera frame; rotations are around the camera’s own axes. Local mode uses the same key map as the remote client (the same wire-protocol strings the GPU server expects).
Focus the main viewport (click the image) before using keyboard shortcuts.
| Input | Action |
|---|---|
| ↑ / ↓ | |
| Move forward / backward (along view axis) | |
| ← / → | |
| Strafe left / right | |
| Shift + ↑ | |
| Ascend along camera-local +Y | |
| Shift + ↓ | |
| Descend along camera-local −Y | |
| W / S | |
| Pitch up / down | |
| A / D | |
| Yaw left / right | |
| Q / E | |
| Roll counter-clockwise / clockwise |
- / − (or = / −) | Zoom out / in | Ctrl + G | Open Go to dialog — teleport camera to a world (x, y, z) | Status bar X Y Z | Edit camera position inline (Enter or focus-out applies) |
| Input | Action |
|---|---|
| Scroll wheel | |
| Move forward / backward (same as + / −) | |
| Click colored link marker in the viewport | |
| Jump to the linked model (loads target PLY) | |
| Hover link marker | Cursor changes to a hand when a link is under the pointer |
| Input | Action |
|---|---|
Drag & drop .ply onto the window |
|
| Load locally or upload to remote server | |
| Browse / Load another… | |
| Pick a PLY from disk | |
| Settings → Connect to server | |
| SSH tunnel + remote WebSocket render server |
Open Session (bottom-right of the status bar) to view models as nodes and links as directed edges.
Press L at a camera pose to create a link to another model. Configure the marker color, label, and target renderer in the dialog; the sphere appears in the viewport and jumps to the linked PLY when clicked.
| Input | Action |
|---|---|
| Left-click a node | |
| Load that model and close the dialog | |
| Right-click a node | |
| Set renderer for this model…, Remove a link…, or Remove node from graph | |
| Right-click an edge | |
| Remove link | |
| L | |
| Create a link at the current camera position to another model (pick target + marker style in the dialog) | |
| Click × on an edge (after Remove a link…) | |
| Delete that link | |
| Load session… / Save session… | |
Read/write a .caddy session file (models + links + per-model renderers) |
| Input | Action |
|---|---|
| Settings → Renderer | |
| Choose the active renderer (local or after remote connect) | |
| 1 – 9, 0 | |
Jump to saved camera preset from camera_presets.json (if present) |
Caddy supports plugin renderers for experimenting with alternate shading
pipelines (e.g. env‑map or GaussianShader‑style models) on top of the same 3DGS
core. Plugins are auto‑discovered at startup from a user_renderers/
folder at the repository root (create it if missing).
A full working example lives in ** user_renderers/render.py** (registers the
user_envmap
renderer with the adapter/wrapper pattern below).Add a Python file, e.g. user_renderers/my_renderer.py
:
from typing import Dict
import torch
from gaussian_splatting.gaussian_renderer import I3DGSRasterizer
from gaussian_splatting.scene.scene_interface import I3DGSScene
class MyRenderer(I3DGSRasterizer):
def __init__(self) -> None:
self._scene: I3DGSScene | None = None
def load_scene(self, ply_path: str) -> I3DGSScene:
raise NotImplementedError
def render(
self,
viewpoint_camera,
scene: I3DGSScene,
bg_color: torch.Tensor,
scaling_modifier: float = 1.0,
override_color: torch.Tensor | None = None,
debug: bool = False,
) -> dict[str, torch.Tensor]:
raise NotImplementedError
def get_renderers() -> Dict[str, callable]:
return {"my_renderer": lambda: MyRenderer()}
Registration: on startup Caddy scans user_renderers/
, imports each .py
,
and calls get_renderers()
. Names are added to
gaussian_splatting.renderer_registry
.
** I3DGSScene** exposes at minimum
means3D
, scales
, rotations
, opacity
;
optional colors
, features
, and extra_attrs
for custom per‑splat data.The built‑in default renderer ** cuda_diff_gaussian** is itself an adapter:
DiffGaussianRasterizerAdapter
implements I3DGSRasterizer
and renders any
I3DGSScene
via diff_gaussian_rasterization
. See
gaussian_splatting/examples/example_scenes.py
for a minimal usage example.If you already have a render function from another 3DGS project (e.g.
GaussianShader) that expects a model object pc
with get_xyz
, get_features
, BRDF fields, etc., you usually need two wrappers:
Scene wrapper (— on…Scene(I3DGSScene)
)load, wrap your native model so Caddy can hold and pass it as a genericI3DGSScene
.Reverse adapter (— on_SceneAdapter
)render, wrap the incomingI3DGSScene
back into the API your legacyrender()
expects.
load_scene(ply)
→ GaussianShaderScene(GaussianModel) # native model → I3DGSScene
UserEnvMapRenderer.render(scene)
→ pc = _SceneAdapter(scene) # I3DGSScene → pc-shaped API
→ render(viewpoint_camera, pc, pipe, …) # your existing code unchanged
Load path (user_renderers/render.py
): UserEnvMapRenderer.load_scene
loads a GaussianShader‑style checkpoint (PLY + optional brdf_mlp/…/brdf_mlp.hdr
)
and returns a GaussianShaderScene
that maps get_xyz
→ means3D
, etc.
Render path: UserEnvMapRenderer.render
calls _SceneAdapter(scene)
so the
generic I3DGSScene
from Caddy looks like a GaussianModel
again, then delegates
to the module’s existing render()
function. A small _ShaderPipe
object stands in for GaussianShader’s pipeline/config.
Test the bundled example:
python scripts/render_one_frame.py `
--ply "path/to/point_cloud.ply" `
--renderer user_envmap `
--out "frame_envmap.png"
You do not have to rewrite your shading code — only implement the two
interface classes (I3DGSScene
wrapper + I3DGSRasterizer
) and a thin adapter
between them and your existing render()
.
Minimal renderer test with a camera at the origin. Use to check the rendering environment.
scripts/render_single_ply.py
python scripts/render_single_ply.py `
--ply "point_cloud.ply" `
--out "test.png"
Required parameters:
--ply
Optional parameters:
--width
--height
--out
scripts/caddy_core.py
— rendering backends (LocalRenderer
,RemoteRenderer
) used by the viewer and headless render scripts.scripts/camera_math.py
— camera/rotation math helpers.
— this package is only needed forNo module named 'diff_gaussian_rasterization'
local GPU rendering. For remote-only viewing you can ignore it; to render locally, install it (seeSection 3).— install the Qt system libs:Could not load the Qt platform plugin "xcb"
/libxcb-*
errors (WSL/Linux)sudo apt install -y libxcb-cursor0 libxcb-icccm4 libxcb-keysyms1
, and if neededexport QT_QPA_PLATFORM=xcb
. Runldd <PySide6>/Qt/plugins/platforms/libqxcb.so | grep "not found"
to see any remaining missing library.— make sure the repo and conda env exist on theFailed to start remote server
/ connection closedserver at the configuredremote project path, and that the SSH target, port, and conda env name inSettings are correct.— setOMP: Error #15 ... libiomp5md.dll already initialized
(Windows)KMP_DUPLICATE_LIB_OK=TRUE
before launching (e.g. PowerShell:$env:KMP_DUPLICATE_LIB_OK="TRUE"
).— use the pinnedDLL load failed while importing QtCore
(Windows)PySide6==6.7.3
fromrequirements.yaml
; newer PySide6 builds can require a newer MSVC runtime than conda ships.