AmpleGCG: Learning a Universal Generative Model for Jailbreaking

Researchers Zeyi Liao and colleagues published AmpleGCG, a generative model that learns the distribution of adversarial suffixes from GCG optimization and generates hundreds of jailbreak suffixes per harmful query in seconds, achieving near 100% attack success rate on Llama-2-7B-chat and Vicuna-7B and 99% on GPT-3.5, according to the arXiv paper (v3, revised 24 Nov 2024). The model produces 200 adversarial suffixes for one harmful query in 4 seconds and transfers from open-source to closed-source LLMs, which the authors say makes defense more challenging.

Computer Science Computation and Language Submitted on 11 Apr 2024 v1 https://arxiv.org/abs/2404.07921v1 , last revised 24 Nov 2024 this version, v3 Title:AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMs View PDF https://arxiv.org/pdf/2404.07921 HTML experimental https://arxiv.org/html/2404.07921v3 Abstract:As large language models LLMs become increasingly prevalent and integrated into autonomous systems, ensuring their safety is imperative. Despite significant strides toward safety alignment, recent work GCG~\citep{zou2023universal} proposes a discrete token optimization algorithm and selects the single suffix with the lowest loss to successfully jailbreak aligned LLMs. In this work, we first discuss the drawbacks of solely picking the suffix with the lowest loss during GCG optimization for jailbreaking and uncover the missed successful suffixes during the intermediate steps. Moreover, we utilize those successful suffixes as training data to learn a generative model, named AmpleGCG, which captures the distribution of adversarial suffixes given a harmful query and enables the rapid generation of hundreds of suffixes for any harmful queries in seconds. AmpleGCG achieves near 100\% attack success rate ASR on two aligned LLMs Llama-2-7B-chat and Vicuna-7B , surpassing two strongest attack baselines. More interestingly, AmpleGCG also transfers seamlessly to attack different models, including closed-source LLMs, achieving a 99\% ASR on the latest GPT-3.5. To summarize, our work amplifies the impact of GCG by training a generative model of adversarial suffixes that is universal to any harmful queries and transferable from attacking open-source LLMs to closed-source LLMs. In addition, it can generate 200 adversarial suffixes for one harmful query in only 4 seconds, rendering it more challenging to defend. Submission history From: Zeyi Liao view email https://arxiv.org/show-email/7f6fdea0/2404.07921 Thu, 11 Apr 2024 17:05:50 UTC 2,460 KB \ v1\ https://arxiv.org/abs/2404.07921v1 Thu, 2 May 2024 01:08:37 UTC 2,292 KB \ v2\ https://arxiv.org/abs/2404.07921v2 v3 Sun, 24 Nov 2024 18:03:43 UTC 2,292 KB References & Citations Loading... Bibliographic and Citation Tools Bibliographic Explorer What is the Explorer? https://info.arxiv.org/labs/showcase.html arxiv-bibliographic-explorer Connected Papers What is Connected Papers? https://www.connectedpapers.com/about Litmaps What is Litmaps? https://www.litmaps.co/ scite Smart Citations What are Smart Citations? https://www.scite.ai/ Code, Data and Media Associated with this Article alphaXiv What is alphaXiv? https://alphaxiv.org/ CatalyzeX Code Finder for Papers What is CatalyzeX? https://www.catalyzex.com DagsHub What is DagsHub? https://dagshub.com/ Gotit.pub What is GotitPub? http://gotit.pub/faq Hugging Face What is Huggingface? https://huggingface.co/huggingface ScienceCast What is ScienceCast? https://sciencecast.org/welcome Demos Recommenders and Search Tools Influence Flower What are Influence Flowers? https://influencemap.cmlab.dev/ CORE Recommender What is CORE? https://core.ac.uk/services/recommender arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs https://info.arxiv.org/labs/index.html .