{"slug": "differential-privacy-for-hugging-face-trainers-without-rewriting-your-training", "title": "Differential Privacy for Hugging Face Trainers – Without Rewriting Your Training Loop", "summary": "JetBrains Research has open-sourced DPTrainer, a library that integrates Opacus with Hugging Face's Trainer to enable differential privacy (DP-SGD) training without rewriting training loops or modifying trainer source code. The tool addresses privacy concerns in LLM training by preventing membership inference attacks, allowing JetBrains to use sensitive IDE-generated data for model training.", "body_md": "## JetBrains Research\n\nResearch is crucial for progress and innovation, which is why at JetBrains we are passionate about both scientific and market research\n\n[Research](/research/category/research/)\n\n# Differential Privacy for Hugging Face Trainers – Without Rewriting Your Training Loop\n\nIt is a well-known problem by now that training LLMs on sensitive data raises serious privacy concerns. In a recent [blog post](https://blog.jetbrains.com/research/2026/06/membership-inference/), we talked about membership inference attacks and our research on mitigating them.\n\nAt JetBrains Research, we are deeply concerned about user privacy and continually developing new methods and tools to improve privacy protection. In this post, we present [DPTrainer](https://jb.gg/dptrainer), a new library we’ve developed and now open-sourced. DPTrainer smoothly integrates Opacus and Hugging Face [Trainer](https://huggingface.co/docs/transformers/en/main_classes/trainer) so that you can train privacy-preserving models without rewriting training loops or modifying trainer source code.\n\n**The importance of differential privacy**\n\nIt’s been widely [observed](https://arxiv.org/pdf/2010.14701) that the quality of a model scales along three axes: size, compute, and data. Larger models offer more capacity but suffer from less efficient training and costlier inference. More compute used during training naturally incurs higher costs and takes more time. The data axis, on the other hand, is mostly constrained by the ability to acquire it in sufficient quality and quantity.\n\nDifferential privacy is our solution to the data-gathering hurdle. Basically, differential privacy is a mathematical framework that protects individual data points used for training. The core guarantee: a model trained with differential privacy behaves almost identically whether or not any single example was included in the training set. For LLMs, which are known to memorize training data and can reproduce it in response to adversarial prompting, this is the strongest known defense against leakage. Even sophisticated [Membership Inference Attacks](https://blog.jetbrains.com/research/2026/06/membership-inference/), given access to model weights, confidence scores, and the base model architecture, cannot determine whether a specific example protected by this method was included in the training set.\n\nIn practice, differential privacy is applied to neural network training through what is known as the differentially private stochastic gradient descent (DP-SGD). Rather than computing a single gradient over the entire batch, the DP-SGD computes one gradient per sample, clips it to bound outliners, aggregates the gradients in the batch and than injects noise making the footprint of any single example indistinguishable.\n\nBy guaranteeing the privacy of our training method, we can exploit previously unavailable channels and use data generated every day through our IDEs (see our data collection [policy](https://www.jetbrains.com/legal/docs/terms/product_data_collection/#detailed-data-collection) and a recent [post](https://blog.jetbrains.com/blog/2025/09/30/detailed-data-sharing-for-better-ai/) on data sharing for AI). This gives us high data quantity due to the size of our user base, as well as high data quality, as the data is generated in the process of writing code, not just extracted from the final product. Such advantages guarantee that our upcoming models will hit above their weight (pun intended).\n\n**The gap it closes**\n\n[Opacus](https://opacus.ai/) is the go-to library for `DP-SGD`\n\nin PyTorch. It provides everything you need: per-sample gradient computation, a `DPOptimizer`\n\n, privacy accountants, and Poisson-sampled data loaders. The catch is that it’s designed around a manual PyTorch training loop, which is inconvenient and not well integrated into the Hugging Face platform.\n\nHugging Face` Trainer `\n\nand Transformers Reinforcement Learning ([TRL](https://huggingface.co/docs/trl/index))’s alignment trainers (e.g. `SFTTrainer`\n\n, `DPOTrainer`\n\n) are the top high-level training APIs for transformers. They handle distributed training, checkpointing, evaluation, callbacks, and many other things you don’t want to reimplement. However, they have zero awareness of differential privacy.\n\nWiring Opacus into a `Trainer`\n\n-based workflow requires touching model wrapping, optimizer creation, data loading, loss computation, checkpointing, and callback management. These interact in subtle ways, and getting any one wrong can break your privacy guarantee, and do it silently.\n\nTo fix this, our researchers Evgeny Grigorenko and David Stanojevic created [DPTrainer](https://jb.gg/dptrainer); and Mihajlo Linic now maintains it. `DPTrainer`\n\nhandles these issues with care.\n\n**A genuine drop-in replacement**\n\nA key concept in differential privacy is the **privacy budget**. This concept represents the maximum theoretical risk of information leakage we are willing to accept. In other words, it is the maximum amount that any single datapoint could shift the output distribution. An important property of the privacy budget is that its expenditure is cumulative, forcing a trade-off between privacy and performance as higher privacy necessitates higher injection of noise into the gradient.\n\n`DPTrainer`\n\nextends `transformers.Trainer`\n\n, and incorporates the privacy budget with an added `PrivacyArguments`\n\ndataclass. Every standard training argument, callback, checkpoint, and evaluation workflow works unchanged, as can be seen in the following code:\n\n``` python\nfrom dptrainer import DPTrainer, PrivacyArguments\n\nprivacy_args = PrivacyArguments(\n    target_epsilon=8.0,\n    per_sample_max_grad_norm=1.0,\n)\n\ntrainer = DPTrainer(\n    model=model,\n    args=training_args,\n    train_dataset=train_dataset,\n    privacy_args=privacy_args,\n    data_collator=data_collator,\n)\n\ntrainer.train()\n```\n\nSet a `target_epsilon`\n\nto match your privacy budget, and `DPTrainer`\n\nwill handle the rest. The internal accountant keeps track of the budget expenditure, and the remaining budget is saved during checkpointing so the run can be easily resumed.\n\n**Privatizing TRL and other specialized trainers**\n\nThe real power comes from `privatize_trainer`\n\n. Many workflows use `Trainer`\n\nsubclasses: e.g. `DPOTrainer`\n\nfor preference learning, `SFTTrainer`\n\nfor instruction tuning, and `Seq2SeqTrainer`\n\nfor generation. These all add task-specific loss functions and generation logic on top of the base class. Rewriting those to inherit from `DPTrainer`\n\nwould be invasive and fragile.\n\n`privatize_trainer`\n\npatches any `Trainer`\n\n-based class at runtime, injecting `DPTrainer`\n\ninto its inheritance chain without touching the class’s own logic:\n\n``` python\nfrom trl import DPOTrainer\nfrom dptrainer import PrivacyArguments, privatize_trainer\n\nprivatize_trainer(DPOTrainer)  # one line\n\ntrainer = DPOTrainer(\n    model=model,\n    args=training_args,\n    train_dataset=train_dataset,\n    processing_class=tokenizer,\n    privacy_args=PrivacyArguments(target_epsilon=8.0, per_sample_max_grad_norm=1.0),\n)\ntrainer.train()\n```\n\nThe patched trainer keeps all its original behavior (e.g. reward computation, DPO loss, generation), while gaining DP-SGD.\n\n**What DPTrainer handles, so you don’t have to**\n\n`DPTrainer`\n\nautomatically manages the following:\n\n**Noise addition.** Adds calibrated Gaussian noise to the aggregated gradients via`DPOptimizer`\n\n.**Gradient clipping.** Clips each sample’s gradient individually before aggregation, not the batch gradient. Supports flat, adaptive (AdaClip), and per-layer strategies via`clipping`\n\nand`per_sample_max_grad_norm`\n\n.**Gradient computation.** Wraps the model in Opacus’s`GradSampleModule`\n\nfor per-sample gradients, which is required for DP-SGD correctness.**Optimizer creation.** Intercepts`create_optimizer`\n\nto wrap the Hugging Face-created optimizer with`DPOptimizer`\n\n.**Data loading.** Overrides`get_train_dataloader`\n\nto return a`DPDataLoader`\n\nwith Poisson sub-sampling, which is what enables privacy amplification by sampling.**Noise calibration.** Given a`target_epsilon`\n\nand your training configuration,`DPTrainer`\n\ncomputes the correct`noise_multiplier`\n\nautomatically – no manual binary search.**Privacy accounting.** A`DPCallback`\n\nhooks into the optimizer step and tracks the running privacy budget after every update.**Checkpointing.** Saves and restores accountant state alongside model weights, so your privacy budget tracking remains correct after resuming.**Early stopping.** A privacy-budget-aware stopping mechanism halts training automatically when the entire budget is exhausted.\n\n**Flexible configuration**\n\n`PrivacyArguments`\n\nexposes the knobs you’d expect:\n\n: set one or the other – they’re mutually exclusive.`target_epsilon`\n\n/`noise_multiplier`\n\n: choose`clipping`\n\n`\"flat\"`\n\n(standard),`\"adaptive\"`\n\n(AdaClip), or`\"per_layer\"`\n\n.: toggle Poisson sub-sampling for privacy amplification.`poisson_sampling`\n\n:`grad_sample_mode`\n\n`\"hooks`\n\n” (default).: privacy accountant type (RDP by default).`accountant`\n\n: log budget expenditure at training steps, eval, both, or not at all.`epsilon_log_mode`\n\n**Try our DPTrainer**\n\nDifferential privacy is increasingly a compliance requirement, not just a research nicety. Regulations around training on personal data, and the growing awareness of membership inference attacks against LLMs, mean that teams need practical, auditable differential privacy training. The hard part has never been the math; it’s been the engineering. And DPTrainer removes that barrier.\n\nIf you’re training transformers on sensitive data and using any part of the Hugging Face ecosystem, this is worth a try.\n\n#### Subscribe to JetBrains Research blog updates", "url": "https://wpnews.pro/news/differential-privacy-for-hugging-face-trainers-without-rewriting-your-training", "canonical_source": "https://blog.jetbrains.com/research/2026/08/dp-trainer/", "published_at": "2026-08-27 15:31:12+00:00", "updated_at": "2026-08-27 16:18:29.795198+00:00", "lang": "en", "topics": ["ai-research", "ai-safety", "ai-tools", "machine-learning"], "entities": ["JetBrains Research", "DPTrainer", "Opacus", "Hugging Face", "PyTorch", "TRL"], "alternates": {"html": "https://wpnews.pro/news/differential-privacy-for-hugging-face-trainers-without-rewriting-your-training", "markdown": "https://wpnews.pro/news/differential-privacy-for-hugging-face-trainers-without-rewriting-your-training.md", "text": "https://wpnews.pro/news/differential-privacy-for-hugging-face-trainers-without-rewriting-your-training.txt", "jsonld": "https://wpnews.pro/news/differential-privacy-for-hugging-face-trainers-without-rewriting-your-training.jsonld"}}