{"slug": "shuffling-is-not-enough-breaking-permutation-based-model-confidentiality", "title": "Shuffling Is Not Enough: Breaking Permutation-Based Model Confidentiality", "summary": "A September 11, 2026 arXiv paper shows that permutation-based model confidentiality in hybrid fully homomorphic encryption (FHE) inference fails, with d+1 admissible queries per linear layer enabling exact recovery of a permutation-invariant layer summary. The authors recovered all linear layers of a SAFHIRE-style ResNet-20 end-to-end from TFHE transcripts with zero error using 5,712 direct queries, and confirmed exact per-layer recovery on pretrained ImageNet-scale CNNs and ViT-B/16. The paper also finds that input differential privacy is orthogonal to model confidentiality and that the local-DP premise for shuffle amplification cannot hold under correctness-bounded noise, while suppressing the leaked spectra destroys inference utility.", "body_md": "# Computer Science > Cryptography and Security\n\n  [Submitted on 11 Sep 2026]\n\n# Title:Shuffling is Not Enough: Breaking Permutation-Based Model Confidentiality in Hybrid FHE Inference\n\n[View PDF](https://arxiv.org/pdf/2609.12911)\n\n[HTML (experimental)](https://arxiv.org/html/2609.12911v1)\n\nAbstract:Hybrid fully homomorphic encryption~(FHE) inference improves the practicality of private inference by letting the server evaluate linear layers homomorphically while the client decrypts and applies nonlinearities. Recent schemes attempt to protect model confidentiality by returning noisy, output-permuted responses and appealing to shuffle-model differential privacy~(DP). We show that this protection fails in the correctness regime required by hybrid FHE systems. For a $d$-input linear layer, $d+1$ admissible queries suffice for exact recovery of a permutation-invariant layer summary, hence for perfect model distinguishability. We further show that input DP is orthogonal to model confidentiality and that the local-DP premise required for shuffle amplification cannot hold under correctness-bounded noise. We recover all linear layers of a \\safhire{}-style ResNet-20 end-to-end from TFHE transcripts with zero error, using $d+1$ queries per layer for a total of $5{,}712$ direct queries. Under the same query model, we also confirm exact per-layer recovery on pretrained ImageNet-scale CNNs and ViT-B/16. The leaked spectra enable fingerprinting, lineage attribution, and improved logit-based extraction, while suppressing them destroys inference utility.\n    \n\n### References & Citations\n\nLoading...\n\n# Bibliographic and Citation Tools\n\nBibliographic Explorer \n\n*(*[What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))\nConnected Papers \n\n*(*[What is Connected Papers?](https://www.connectedpapers.com/about))\nLitmaps \n\n*(*[What is Litmaps?](https://www.litmaps.co/))\nscite Smart Citations \n\n*(*[What are Smart Citations?](https://www.scite.ai/))\n# Code, Data and Media Associated with this Article\n\nalphaXiv \n\n*(*[What is alphaXiv?](https://alphaxiv.org/))\nCatalyzeX Code Finder for Papers \n\n*(*[What is CatalyzeX?](https://www.catalyzex.com))\nDagsHub \n\n*(*[What is DagsHub?](https://dagshub.com/))\nGotit.pub \n\n*(*[What is GotitPub?](http://gotit.pub/faq))\nHugging Face \n\n*(*[What is Huggingface?](https://huggingface.co/huggingface))\nScienceCast \n\n*(*[What is ScienceCast?](https://sciencecast.org/welcome))\n# Demos\n\n# Recommenders and Search Tools\n\nInfluence Flower \n\n*(*[What are Influence Flowers?](https://influencemap.cmlab.dev/))\nCORE Recommender \n\n*(*[What is CORE?](https://core.ac.uk/services/recommender))\n# arXivLabs: experimental projects with community collaborators\n\narXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.\n\nBoth individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.\n\nHave an idea for a project that will add value for arXiv's community? [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html).", "url": "https://wpnews.pro/news/shuffling-is-not-enough-breaking-permutation-based-model-confidentiality", "canonical_source": "https://arxiv.org/abs/2609.12911", "published_at": "2026-09-15 04:07:08+00:00", "updated_at": "2026-09-15 04:33:46.382906+00:00", "lang": "en", "topics": ["ai-safety", "ai-research", "machine-learning", "neural-networks", "ai-ethics"], "entities": ["arXiv", "SAFHIRE", "ResNet-20", "TFHE", "ImageNet", "ViT-B/16"], "alternates": {"html": "https://wpnews.pro/news/shuffling-is-not-enough-breaking-permutation-based-model-confidentiality", "markdown": "https://wpnews.pro/news/shuffling-is-not-enough-breaking-permutation-based-model-confidentiality.md", "text": "https://wpnews.pro/news/shuffling-is-not-enough-breaking-permutation-based-model-confidentiality.txt", "jsonld": "https://wpnews.pro/news/shuffling-is-not-enough-breaking-permutation-based-model-confidentiality.jsonld"}}