perturber: A Stateless Service for Semantics-Preserving Text Perturbations

Researchers released perturber, a stateless service and Python library that applies exactly one semantics-preserving perturbation per call, drawn from a first-party catalog of 68 perturbations across 11 families, according to the arXiv paper arXiv:2610.12667v1. The tool is designed so the service and the library produce identical output by construction, with composition left to the caller chaining calls, and it measures meaning-preservation per perturbation rather than assuming it. The release includes a live playground, an introspectable catalog for benchmark selection, and a one-command reproducible evaluation, addressing the problem that existing perturbation tools are coupled to a benchmarking stack rather than packaged as a reusable transform layer.

arXiv:2610.12667v1 Announce Type: new Abstract: Large language model LLM evaluations on clean benchmarks overstate model robustness: real inputs arrive perturbed, and these meaning-preserving variants routinely move model answers. Studying this systematically requires applying perturbations one at a time, reproducibly, and identically wherever they run, yet existing tools couple them to a benchmarking stack rather than packaging them as a reusable transform layer. We present perturber, a stateless service and Python library that applies exactly one semantics-preserving perturbation per call, drawn from a first-party catalog of 68 perturbations across 11 families. It turns on a single primitive, with composition left to the caller chaining calls, so the service and the library produce identical output by construction. We evaluate the tool itself, its correctness, reproducibility, and cost, rather than any model's robustness, measuring meaning-preservation per perturbation rather than assuming it. The demonstration adds a live playground, an introspectable catalog for benchmark selection, and a one-command reproducible evaluation.