Neural block texture compression with CUDA

The 2 latent layers (containing the per-block parameters) are sampled with nearest sampling.

• Format: per-pixel selectors + one latent per 4×4 block + tiny MLP block decoder
• Selectors: 1024×1024×2, 3 bits + 1 bit per pixel, picked by exact exhaustive search each iteration
• Block latent: 256×256×4, 8 bits/value, trained by ES with footprint attribution
• Decoder: 8→36→36→3 MLP, 1767 weights; inputs = 2 selectors + 4 block values + 2 block-local coords
• Decoder training: antithetic ES (64 pairs, 4096-pixel minibatches) then central finite differences for the last quarter
• Latent training: 4 antithetic pairs/iteration on the full image, no backprop anywhere
• 3000 iterations, lr annealed 1× → 0.05× over the second half
• 40.78 dB fp32, 40.71 dB with the block latent quantized to 8 bits
• 6.03 bpp raw (4 bpp selectors + 2 bpp block latent + 0.05 bpp MLP), 5.54 bpp entropy-coded
• 96 bits per 4×4 block: 64 selector bits + 32 latent bits
• 55 s total on an RTX 5090, everything on the GPU (CUDA 13.1)

Left: original, right: reconstruction  The block artifacts are there if you look closely: