Neural block textures vs. GPU textures (like BC7) at Equal Bitrates

Developer Rich Geldreich released neural block textures, a GPU texture compression method that replaces the fixed hardware decoder with a small configurable MLP (roughly 250-1200 weights) and PVRTC1-style bilinear endpoint sampling. In a matched-bitrate test on a 1080x1080 photo at 8.0 bpp, the neural format reached 43.15 dB versus 43.78 dB for the analytical bc7f encoder and 41.03-45.59 dB across bc7e scalar levels. Geldreich reports the approach is implemented and working with CUDA training, scales easily to multi-channel PBR materials, and could plausibly be built into GPU hardware.

The GitHub release is here https://github.com/richgel999/neural block textures . The existing neural texture work has made the entire space overly complicated. The first step beyond GPU textures is to essentially keep the current GPU texture-style latent, but replace the fixed hardware generative decoder with a configurable MLP, and adopt PVRTC1-like block which is at 1/4 resolution in PVRTC1 endpoint bilinear sampling: Replace the classical GPU texture decoder ASTC/BC1-7 etc. with a neural net MLP . Now you get as many output channels as needed. Then bilinear filter the "color" endpoints like PVRTC1 , and use a flexible say 2-4 of channels. I've been testing the "colors" latent which is typically 2-4 channels at 1/4, 1/6, 1/8 and 1/16 sizes relative to the texture's resolution. Per-texel "selectors"/"weights" can be unchanged: use 1 channel for single textures, 2-3 for materials. I've tested 1-5 bits per channel, with potentially a different number of bits on each channel. The "encoder" is the training step which can use backprop or ES Evolution Strategies etc. It's amazingly flexible and looks remarkable. ES on the per-texel or lower res latents is easily optimized https://github.com/richgel999/neural tex block compression , and at this tiny MLP size it's not a big deal. Even on a single texture not a material , the bitrate and quality is roughly competitive vs. transform methods on GPU texture latents. Except this method scales easily to materials and inference decoding cost is amortized across the material. The more channels the MLP outputs, the lower the effective bitrate. Around 250-1200 MLP weights seems like a good range for this method. Tiny networks. I don't see why this can't be implemented directly in GPU hardware. For inference on load on the CPU this approach seems entirely reasonable, using threading and SIMD to accelerate inference. No sidebands like NTC uses are needed for fast full-format BC7 encoding - the math is simple, and is already solved by bc7f https://github.com/BinomialLLC/basis universal/wiki/Transcoder-Internals-Analytical-Real-Time-Encoders-and-Transcoders bc7f . None of this is theoretical - I have this working now using CUDA for training with a CPU SIMD fallback if needed . It's exciting. Some hard numbers, comparing this neural texture design vs. BC7, are from this X post https://x.com/richgel999/status/2096742398093144311 : A single complex 1080x1080 photo, neural block texture vs. two BC7 encoders at matched 8.0 bpp bitrates neural was 8.02, including quantized latents+all MLP weights as FP16 : Neural block texture: 43.15 dB bc7f analytical: 43.78 dB bc7e scalar level 0: 41.03 dB bc7e scalar level 1: 44.85 dB bc7e scalar level 2: 44.67 dB bc7e scalar level 3: 45.18 dB bc7e scalar level 6 slowest : 45.59 dB Neural config: Per-texel "selectors" latent: 2 channels, 4+2 bits 6 bits total 1/4 resolution "colors" latent: 4 channels, 8-bit channels 32 bits total MLP: 1,083 weights Training: Quant aware, ES+CD, 8k iterations, 106 secs on a 5090 This is a sanity check: at equal very high bitrates, can this format and my trainer match a highly tuned widely deployed BC7 encoder? Yes it can, or very close. Unlike any GPU texture format, this design is highly configurable/flexible. It easily scales down to low bitrates, or scales wider to handle multi-channel PBR textures with the same latents.