We present Isekai3D SDG (Synthetic Data Generation), a platform for generating diverse 3D gameplay data for world models, computer vision, neural graphics, and gameplay agents. The platform combines reusable environments and characters with modular Unreal Engine plugins. A two-stage workflow records world simulation and then replays it to capture complementary views and rendering passes from the same underlying gameplay. Configurable cameras, resolutions, and frame rates support paired observations with frame-aligned camera, character, and action metadata. Outputs include RGB, depth, surface normals, and instance IDs. A packaged application supports automated data generation without installing the Unreal Engine editor or build tools, while custom plugins and compatible DLL extensions let researchers adapt individual components.
To cite our work, use:
@techreport{karp2026isekai3d,
title = {{Isekai3D SDG}: A Platform for Diverse Synthetic Gameplay Visual Data},
author = {Karp, Rafa{\l} and Liu, Gene and Li, Shih-Hsin and Courtemanche, Eric and Wang, Shengze and Spljut, Josef and Pankratz, David and Kolbe, Mischa and Stengel, Michael and Elflein, Sven and Grygielski, Adam and Wu, Liwen and Patney, Anjul},
institution = {NVIDIA},
year = {2026},
month = oct,
url = {<when we have url to uploaded .pdf>},
type = {Technical Report}
}