{"slug": "opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network", "title": "OpenDLSS: A Vulkan Reimplementation of Nvidia's DLSS 5 Neural Rendering Network", "summary": "A developer released OpenDLSS-NR, a Vulkan reimplementation of Nvidia's DLSS 5 Neural Rendering network that runs bit-exact against the original, matching all 75 block boundaries byte for byte. The project reproduces the same 71-block Swin/ViT network as DLSS-NR build 310.8.0, running FP8 (E4M3) activations with FP16 accumulation and 141 MiB of weights, and includes a second independent WebGPU browser port that runs without tensor cores or FP8. On an RTX 4070 SUPER the network runs 241 dispatches per frame at 2.8 ms for 768x768, 7.8 ms for 1920x1080, 12.6 ms for 2560x1440 and 29.3 ms for 3840x2160, with users supplying their own model weights.", "body_md": "A Vulkan reimplementation of NVIDIA's DLSS 5 Neural Rendering network, bit-exact against the original.\n\nThe same 71-block Swin / ViT network as DLSS-NR build 310.8.0, running FP8 on the tensor cores. The intermediates match too, not just the final image: all 75 block boundaries, byte for byte.\n\n`ports/browser-webgpu/` is a second, independent implementation: the same bytes in a browser, with no\ntensor cores and no FP8.\n\n**You supply the weights**, as a model directory in the layout described below.\n\nA U-net of shifted-window transformer blocks with a global ViT at the bottom: 71 blocks over six pooling levels, FP8 (E4M3) activations with FP16 accumulation, 141 MiB of weights. It is a generative neural rendering network (NVIDIA's term): it re-renders the frame the engine already drew, generating detail from injected noise and adjusting tone, structure and skin under a style setting. Input and output are the same resolution; it is not an upscaler.\n\n*The WebGPU port at 2048x1152, NR off on the left and on on the right. Scene:\n[Cowboy Gramps](https://www.blendkit.com/asset-gallery-detail/96dce188-9c9c-4699-a45a-48663fbbbcb7/) by\nMuhammed Ismayil, CC0.*\n\nIt takes one rendered frame (a low dynamic range proxy of it, three lanes of Gaussian noise, the previous\nframe's output reprojected, and five conditioning scalars) and produces four f32 channels per pixel: an RGB\nresidual and one temporal-blend logit. [docs/network.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/network.md) is the graph in full. NVIDIA\ndescribes the model in its report,\n[DLSS 5: Generative Neural Rendering](https://research.nvidia.com/labs/adlr/DLSS5/files/DLSS5_Report.pdf)\n([project page](https://research.nvidia.com/labs/adlr/DLSS5/)).\n\n```\npowershell -File scripts\\fetch_tools.ps1 [-Npm]     # once: tools\\ (glslang, Vulkan-Headers, volk, CMake, Ninja)\npowershell -File scripts\\build.ps1                  # shaders, PTX, build\\dlss5vk.exe\npowershell -File scripts\\fetch_filament.ps1         # once, for the demo: third_party\\filament (+ the patch)\npowershell -File scripts\\build_filament.ps1         # once, for the demo: third_party\\filament-install\npowershell -File scripts\\build_demo.ps1             # build\\demo\\dlss5-demo.exe\nbuild\\dlss5vk.exe bench   --model <dir> --width 768 --height 768\nbuild\\dlss5vk.exe profile --model <dir> --width 768 --height 768   # per-dispatch timings\nbuild\\dlss5vk.exe parity  --model <dir> --fixture <dir>            # bit-exactness against a fixture\nbuild\\dlss5vk.exe verify  --model <dir> --fixture <dir>            # block-0 kernel-by-kernel bisect\npython scripts\\ptx\\test_fast_divmod.py                             # the PTX divider, over every n < 2^24 (numpy)\n```\n\nThe demo can be double-clicked. It lists every scene under `build\\scenes` in the **Demo scene** dropdown and\nstarts on the first one, or loads the glTF given on the command line. The model directory is `--model <dir>`,\nelse `DLSS5VK_MODEL`, else `models\\nr` next to this README. See [demo/README.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/demo/README.md) for the\nrenderer, the keys, the scenes and `view.json`.\n\nRTX 4070 SUPER, whole network per frame, minimum over 40 frames. 241 dispatches at every resolution.\n\n| resolution | time | \n|---|---|\n| 768x768 | 2.8 ms | \n| 1920x1080 | 7.8 ms | \n| 2560x1440 | 12.6 ms | \n| 3840x2160 | 29.3 ms | \n\nThe GPU alternates between two clock states under sustained load, so medians run a few percent higher. Compare minima.\n\n| Part | Files | Notes | \n|---|---|---|\n| Host | `src/` (C++20) | Vulkan context, model loading and weight re-layout, kernel wrappers, the network graph, a CPU reference of the arithmetic, the `dlss5vk` tool | \n| GLSL kernels | `shaders/` | The reference route: cooperative-matrix FP8 GEMMs, fused 32-channel block, fused QKV + window attention, expert MLP, global attention, elementwise ops. Exact and complete on their own. | \n| PTX kernels | `scripts/ptx/` | Python generators emitting PTX for the fast route: `mma.sync` E4M3 with f16 accumulation, cp.async rings, barrier-free chaining through device counters, split-K GEMMs, streamed global attention. Generated into`build/ptx` by the build. | \n| Demo | `demo/` ,`third_party/filament.patch` | The network inside a Filament (Apache-2.0) frame: Filament patched for per-object motion vectors and a Vulkan interop hook, glTF scenes through gltfio, ImGui controls. | \n| WebGPU port | `ports/browser-webgpu/` | The same network in a browser, bit-exact against the same captures, with no tensor core, no FP8, no fusion between blocks and no chaining: the exactness is in the specification, not in the hardware. 72 ms at 512x512 against 2.7 ms here. | \n\nNot implemented: DLSS-SR, which is a different network. The temporal path is implemented, but in the demo: the\nnetwork's history input lanes and its per-pixel blend logit drive a reprojected feedback loop\n([docs/frame.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/frame.md)). The `dlss5vk` tool runs single frames with no history, which is what the\nreference captures were made with.\n\n- Windows, an NVIDIA Ada (or newer) GPU and a driver exposing `VK_KHR_cooperative_matrix` ,`VK_NV_cooperative_matrix2` ,`VK_EXT_shader_float8` and`VK_NV_cuda_kernel_launch` .\n- Visual Studio 2022 or later with the C++ x64 toolset (any edition or the Build Tools; found through vswhere,\nor set `VCVARS64` to your`vcvars64.bat` ), git, Python 3 for the PTX generators, and Node.js + npm and\nPillow for the scene converter.\n- The portable toolchain under `tools/` (git-ignored):`scripts\\fetch_tools.ps1` downloads glslang 16.6.0,\nVulkan-Headers v1.4.363, volk (pinned tags), CMake 3.31 and Ninja 1.13. No Vulkan SDK install is needed.`-Npm` also installs the scene converter's modules into`tools\\gltf` .\n- For the demo: Filament v1.77.0, cloned and patched by `scripts\\fetch_filament.ps1` and built once by`scripts\\build_filament.ps1` (both git-ignored; about 15 minutes and 6 GB of build tree, placed in`%LOCALAPPDATA%\\dlss5-vulkan` or`DLSS5_FILAMENT_BUILD_DIR` ;`DLSS5_BUILD_JOBS` caps the parallel\ncompiles, default 8, because MSVC takes up to a GB per job on Filament).\n\n`nr::Model` reads `manifest.json`: a `stages` array (each entry: `id`, `file` relative to the directory,\n`packedByteLength`, `sha256`) and a `tensors` array (each entry: `name`, `block`, `layer`, `parameter`,\n`stage`, `stageOffset`, `byteLength`). Stage files hold the E4M3 weights as packed bytes; the host re-lays\nthem out into the matrix forms the kernels consume (`src/nr_model.cpp`). Nothing in this repository produces\nsuch a directory.\n\nThe graph is the 71-block network of 310.8.0 and nothing else: a model with a different block count is refused at load.\n\n`parity` compares against recorded captures of the original, which are not part of this repository. A fixture is a\ndirectory with a `manifest.json`:\n\n| key |  | \n|---|---|\n| `sourceDimensions` ,`fullDimensions` | the valid size and the padded field | \n| `proxy`*or*` inputFeatures` | the input: an RGBA f32 image (with `conditioning` ,`seed` ,`autoMask` ), or the f32 features themselves | \n| `checks` | what the fixture gates, any of `\"boundaries\"` ,`\"head\"` ,`\"output\"` ; required and never empty | \n| `blocks` ,`transitions` | `\"boundaries\"` : E4M3 references (`block` /`id` ,`width` ,`height` ,`channels` ,`file` ) | \n| `omittedBoundaries` | `\"boundaries\"` :`{name: reason}` for each comparable boundary the fixture has no reference for | \n| `referenceHead` | `\"head\"` : the f32 RGBA head | \n| `nativeOutput` | `\"output\"` : the composed image,`dtype``\"f32\"` (RGBA halves, needs`proxy` ) or`\"u8\"` (an 8-bit capture) | \n\nEverything is validated before the GPU runs, and a fixture that fails any of it is refused: a declared check without\nits reference, a reference that is missing, short or names nothing in the graph, a reference no declared check uses,\nor a comparable boundary (blocks 0-69, the five encoder transitions) with neither a reference nor a reason. Verdicts\nare **bit-exact** (the pass), **equal only up to the sign of zero** (a failure), **within one code** (the 8-bit\ncapture only, reported apart) or a mismatch; see [docs/numerics.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/numerics.md). The head and the output are\ncompared on the production schedule, resubmitted `--repeat` times (default 3), which must also agree with the same\ngraph under barriers; the boundaries come from an instrumented run, whose head must agree with production's. `verify`\nadditionally needs `inputFeatures` and a `block-0` reference.\n\nAll default to the fast, exact route. Every switch keeps the output byte-identical, and `parity` under each of\nthem is part of the gate. Any switch that sends a kernel back to GLSL also turns counter chaining off, because\nonly the PTX kernels take part in it.\n\n- `DLSS5VK_UNFUSED=1` runs the GLSL reference route, up to 2560x1440: it materializes every intermediate.\n- `DLSS5VK_PTX_DIR` is the PTX directory, default`build/ptx` .\n- `DLSS5VK_CHAIN=0` puts barriers between every launch instead of counter chaining.\n- `DLSS5VK_PTX_GEMM` ,`GEMMT` ,`GEMMV` ,`BLOCK32` ,`FFN` ,`QKV` and`ATTN` set to 0 take one kernel family\nback to its GLSL spelling.\n- `DLSS5VK_ATTN_STREAM` forces streamed global attention off (0) or on (1).\n- `DLSS5VK_SPLITK=0` ,`DLSS5VK_VIT_CHAIN=0` and`DLSS5VK_NO_PTX_MLP=1` disable split-K, the ViT chain and the\nPTX MLP.\n- `DLSS5VK_NO_FUSE_PRE` ,`POOL` ,`UPRES` and`POST` set to 1 drop one fusion each.\n- `DLSS5VK_CHAIN_MASK` is a bit mask: 1 expert stages, 2 c32 blocks, 4 split GEMMs. Default 3.\n- `DLSS5VK_DEFER_MAX` is the widest stage whose projection GEMM is fused. Default 128.\n- `DLSS5VK_VALIDATION=1` runs under the Khronos validation layer (refused if it is not installed: a Vulkan SDK, or`VK_LAYER_PATH` at a build of Vulkan-ValidationLayers); an error it reports fails the run.`DLSS5VK_DEBUG=1` only prints the driver's own messages (the PTX compiler's among them).`DLSS5VK_LIST_EXTENSIONS=1` lists\nextensions.\n\n[docs/README.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/README.md) is the index: [network.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/network.md) is the graph,\n[numerics.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/numerics.md) the exactness contract, [weights.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/weights.md) the layouts,\n[execution.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/execution.md) the scheduling, and [frame.md](https://github.com/maanHimself/OpenDLSS-NR/blob/main/docs/frame.md) the demo's frame.\n\nThis project is not affiliated with, endorsed by, or supported by NVIDIA. It contains no NVIDIA software, weights, headers, or instructions for obtaining them. No rights under any NVIDIA intellectual property are granted or implied by this repository or its license, and you are responsible for the licenses that apply to whatever model data you use with it.\n\nMIT for everything in this repository ([LICENSE](https://github.com/maanHimself/OpenDLSS-NR/blob/main/LICENSE)). Third-party components are listed in\n[NOTICE](https://github.com/maanHimself/OpenDLSS-NR/blob/main/NOTICE).", "url": "https://wpnews.pro/news/opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network", "canonical_source": "https://github.com/maanHimself/OpenDLSS-NR", "published_at": "2026-09-30 08:43:21+00:00", "updated_at": "2026-09-30 08:49:02.726499+00:00", "lang": "en", "topics": ["artificial-intelligence", "neural-networks", "generative-ai", "ai-research", "ai-infrastructure"], "entities": ["Nvidia", "OpenDLSS-NR", "DLSS 5 Neural Rendering", "Vulkan", "WebGPU", "RTX 4070 SUPER", "DLSS-NR build 310.8.0", "Muhammed Ismayil"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network", "markdown": "https://wpnews.pro/news/opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network.md", "text": "https://wpnews.pro/news/opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network.txt", "jsonld": "https://wpnews.pro/news/opendlss-a-vulkan-reimplementation-of-nvidia-s-dlss-5-neural-rendering-network.jsonld"}}