{"slug": "split-and-scale-latent-3d-representations", "title": "Split-and-Scale Latent 3D Representations", "summary": "Researchers Dorian Chan, Muhammed Kocabas, Xiaoming Zhao, Oncel Tuzel and Jen-Hao Rick Chang published a paper on split-and-scale, an inference-time method that expands the latent capacity of 3D representations without raising training cost, by applying a single pretrained tokenizer to arbitrarily small sub-regions of a scene. The authors report that extensive ablations show the approach achieves reconstruction quality competitive with much larger models that exceed single-GPU memory limits, and that an autoregressive 3D generator trained on the representation performs competitively with state-of-the-art baselines.", "body_md": "[content type paper](https://machinelearning.apple.com/research/)published November 2026\n\nSplit-and-Scale Latent 3D Representations\n\nAuthorsDorian Chan, Muhammed Kocabas, Xiaoming Zhao, Oncel Tuzel, Jen-Hao Rick Chang\n\nLatent 3D representations have significantly expanded the capabilities of modern shape modeling, enabling compact encoding of 3D scenes and supporting powerful 3D generative models. Despite this progress, current representations struggle to capture the fine details of complex objects, fundamentally limited by the capacity of their latents. However, scaling latent capacity at training time is computationally expensive and often infeasible under memory limits, and naively increasing the number of latents at inference fails to make effective use of the added capacity. In this paper, we address both challenges with split-and-scale, a method that effectively expands latent capacity at inference while keeping training cost tractable. Our key observation is that by representing 3D scenes as continuous functions of 3D coordinates, a single pretrained tokenizer can be applied at inference to arbitrarily small sub-regions of a scene, allocating the same latent capacity to a smaller spatial extent and thus capturing far higher detail. We validate this insight by conducting extensive ablations to study the trade-offs between compute, memory, latent capacity, and reconstruction quality, demonstrating our inference-time scaling method achieves quality competitive with much larger models that exceed single-GPU memory limits. We further demonstrate that split-and-scale supports high-quality generative modeling, training an autoregressive 3D generator that performs competitively with state-of-the-art baselines.\n\nHigh-fidelity image-to-3D generation requires a 3D representation that captures both geometry and appearance. To support relighting and integration into standard rendering pipelines, the representation should include physically based rendering (PBR) modalities such as albedo, metallic-roughness, and surface normals. We propose Luce, a 3D representation that unifies geometry and PBR materials within a voxelized multimodal Gaussian cloud, using…\n\nGAUDI: A Neural Architect for Immersive 3D Scene Generation\n\nSeptember 27, 2022[research area Computer Vision](https://machinelearning.apple.com/research/?domain=Computer%20Vision), [research area Methods and Algorithms](https://machinelearning.apple.com/research/?domain=Methods%20and%20Algorithms)[conference NeurIPS](https://machinelearning.apple.com/research/?event=NeurIPS)\n\nWe introduce GAUDI, a generative model capable of capturing the distribution of complex and realistic 3D scenes that can be rendered immersively from a moving camera. We tackle this challenging problem with a scalable yet powerful approach, where we first optimize a latent representation that disentangles radiance fields and camera poses. This latent representation is then used to learn a generative model that enables both unconditional and…", "url": "https://wpnews.pro/news/split-and-scale-latent-3d-representations", "canonical_source": "https://machinelearning.apple.com/research/split-scale-latent-3d", "published_at": "2026-11-16 00:00:00+00:00", "updated_at": "2026-11-17 23:12:25.758201+00:00", "lang": "en", "topics": ["computer-vision", "machine-learning", "ai-research", "generative-ai"], "entities": ["Dorian Chan", "Muhammed Kocabas", "Xiaoming Zhao", "Oncel Tuzel", "Jen-Hao Rick Chang", "split-and-scale", "Luce", "GAUDI"], "also_reported_by": [], "takeaway": "Applying one pretrained 3D tokenizer to arbitrarily small sub-regions at inference allocates the same latent capacity to a smaller spatial extent, reaching reconstruction quality competitive with much larger models that exceed single-GPU memory limits.", "recommendation": {"advice": "Scale latent capacity at inference, not training: reuse a single pretrained tokenizer on smaller scene sub-regions to gain detail without exceeding single-GPU memory.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/split-and-scale-latent-3d-representations", "markdown": "https://wpnews.pro/news/split-and-scale-latent-3d-representations.md", "text": "https://wpnews.pro/news/split-and-scale-latent-3d-representations.txt", "jsonld": "https://wpnews.pro/news/split-and-scale-latent-3d-representations.jsonld"}}