Distributed and Private Textual Data Synthesis from Embeddings

Researchers Ergute Bao (Inria), Hongyan Chang and Ting Yu (Mohamed bin Zayed University of Artificial Intelligence), Ali Shahin Shamsabadi (Brave Software), and Xiaokui Xiao (National University of Singapore) proposed a differentially private text synthesis method that needs no trusted curator and only lightweight user participation, releasing a one-time DP summary of embedding-space centroids for training-free, non-iterative offline synthesis. The DP-cryptography co-design adds semantic support protection to avoid semantic neighborhoods of infrequent texts, reducing exposure of rare user data, and achieves utility comparable to the state-of-the-art centralized DP synthesis method on four benchmarks.

Distributed and Private Textual Data Synthesis from Embeddings Ergute Bao Inria , Hongyan Chang Mohamed bin Zayed University of Artificial Intelligence , Ali Shahin Shamsabadi Brave Software , Ting Yu Mohamed bin Zayed University of Artificial Intelligence , and Xiaokui Xiao National University of Singapore | Differential Privacy https://brave.com/research/topic/differential-privacy/ , Cryptography https://brave.com/research/topic/cryptography/ , LLM https://brave.com/research/topic/llm/ We revisit differentially private DP text synthesis in the realistic setting of distributed users, where privacy concerns preclude a trusted curator with access to raw user texts. Existing DP text synthesis pipelines are designed for a trusted, centralized curator and often cannot be deployed in distributed settings due to unrealistic trust and access assumptions; when adapted naively, they require repeated, tightly synchronized user participation and incur significant overhead. To address this gap, we propose a DP–cryptography co-design for textual data synthesis that requires no trusted curator and requires only lightweight user participation. Our approach has two optimized components. First, we design a distributed-friendly DP synthesis algorithm that releases a one-time DP summary in an embedding space: it identifies frequent semantic regions and releases their DP centroids, enabling training-free, non-iterative offline text synthesis. We further introduce semantic support protection, which ensures the released summary avoids semantic neighborhoods of infrequent texts, reducing the risk of exposing rare user data. Second, we develop a custom secure protocol that implements this algorithm over distributed user data, enforcing end-to-end DP guarantees without requiring a trusted curator. On four benchmarks, we achieve utility comparable to the state-of-the-art centralized DP synthesis method.