{"slug": "fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil", "title": "Fitting Gemma 4 E2B and an SDXL model on one iPhone: notes from building Anvil", "summary": "A developer building an iPhone app that runs Stable Diffusion through Core ML reported that Apple's ml-stable-diffusion guidance makes the biggest difference for fitting large models on-device, citing 6-bit palettization as Apple's recommended floor for SD weights and Mixed-Bit Palettization recipes that mix 1/2/4/6/8-bit layers to average around 4 bits. The developer noted Apple publishes precomputed mixed-bit recipes for SDXL base on Hugging Face at apple/coreml-stable-diffusion-mixed-bit-palettization, and that Apple's FAQ puts the peak-RAM saving from 16-bit to 4-bit weights at up to 75% on iOS 17 and later, where palettized weights can be decompressed just in time. The notes also cover Apple's SPLIT_EINSUM attention recommendation for the Neural Engine, its advice against SPLIT_EINSUM_V2 for SDXL because compilation takes too long, shipping .mlmodelc over .mlpackage for caching, and reduceMemory loading and unloading the text encoder, UNet and VAE just in time.", "body_md": "Congrats on 4.1, and thanks for writing up the memory side. I work on an iPhone app that also runs Stable Diffusion through Core ML, and these are the parts of Apple’s ml-stable-diffusion README that made the most difference:\n\n- 6-bit palettization is the floor Apple recommends for SD weights. Going lower without visible damage needs their Mixed-Bit Palettization recipes, which mix 1/2/4/6/8-bit layers to land around 4 bits on average; they publish precomputed recipes for SDXL base on Hugging Face (apple/coreml-stable-diffusion-mixed-bit-palettization).\n- On iOS 17 and later, palettized weights can be decompressed just in time instead of at load. Their FAQ puts the peak-RAM saving from 16-bit to 4-bit at up to 75%.\n- For the Neural Engine on iPhone they recommend SPLIT_EINSUM attention. Their SD 1.x/2.x iPhone benchmarks use SPLIT_EINSUM_V2, but they advise against V2 for SDXL because compilation takes far too long.\n- Ship .mlmodelc rather than .mlpackage: the compiled model gets cached, while an .mlpackage re-runs model preparation on every load.\n- reduceMemory on StableDiffusionPipeline loads and unloads the text encoder, UNet and VAE just in time, which is the same idea as your chat/image model swap, one level down.\n\nHave you noticed quality differences between Neural Engine and GPU output at 1024? Apple mentions small differences across compute units and I’ve wondered how visible they get with SDXL.", "url": "https://wpnews.pro/news/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil", "canonical_source": "https://discuss.huggingface.co/t/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil/190518#post_2", "published_at": "2026-11-05 18:43:41+00:00", "updated_at": "2026-11-05 18:52:39.698296+00:00", "lang": "en", "topics": ["ai-tools", "machine-learning", "ai-infrastructure"], "entities": ["Apple", "Stable Diffusion", "Core ML", "ml-stable-diffusion", "SDXL", "Hugging Face", "iPhone", "Neural Engine"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil", "markdown": "https://wpnews.pro/news/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil.md", "text": "https://wpnews.pro/news/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil.txt", "jsonld": "https://wpnews.pro/news/fitting-gemma-4-e2b-and-an-sdxl-model-on-one-iphone-notes-from-building-anvil.jsonld"}}