Document detail
ID

oai:arXiv.org:2408.10701

Topic
Computer Science - Computation and...
Author
Pala, Tej Deep Toh, Vernon Y. H. Bhardwaj, Rishabh Poria, Soujanya
Category

Computer Science

Year

2024

listing date

8/28/2024

Keywords
mutations rainbow prompt adversarial
Metrics

Abstract

In today's era, where large language models (LLMs) are integrated into numerous real-world applications, ensuring their safety and robustness is crucial for responsible AI usage.

Automated red-teaming methods play a key role in this process by generating adversarial attacks to identify and mitigate potential vulnerabilities in these models.

However, existing methods often struggle with slow performance, limited categorical diversity, and high resource demands.

While Rainbow Teaming, a recent approach, addresses the diversity challenge by framing adversarial prompt generation as a quality-diversity search, it remains slow and requires a large fine-tuned mutator for optimal performance.

To overcome these limitations, we propose Ferret, a novel approach that builds upon Rainbow Teaming by generating multiple adversarial prompt mutations per iteration and using a scoring function to rank and select the most effective adversarial prompt.

We explore various scoring functions, including reward models, Llama Guard, and LLM-as-a-judge, to rank adversarial mutations based on their potential harm to improve the efficiency of the search for harmful mutations.

Our results demonstrate that Ferret, utilizing a reward model as a scoring function, improves the overall attack success rate (ASR) to 95%, which is 46% higher than Rainbow Teaming.

Additionally, Ferret reduces the time needed to achieve a 90% ASR by 15.2% compared to the baseline and generates adversarial prompts that are transferable i.e. effective on other LLMs of larger size.

Our codes are available at https://github.com/declare-lab/ferret.

Pala, Tej Deep,Toh, Vernon Y. H.,Bhardwaj, Rishabh,Poria, Soujanya, 2024, Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique

Document

Open

Share

Source

Articles recommended by ES/IODE AI

Comparison between Dual-Energy CT and Quantitative Susceptibility Mapping in Assessing Brain Iron Deposition in Parkinson Disease
nigra substantia healthy depositions p < 05 nucleus brain susceptibility ct bilateral dual-energy iron quantitative mapping values magnetic globus pallidus
Integration of human papillomavirus associated anal cancer screening into HIV care and treatment program in Pakistan: perceptions of policymakers, managers, and care providers
hpv hiv msm transgender women anal cancer screening integration pakistan system managers pakistan informants anal screening cancer lack healthcare hiv