{"id":"arxiv:2305.17444","kind":"preprint","title":"Query-Efficient Black-Box Red Teaming via Bayesian Optimization","authors":["Deokjae Lee","JunYeong Lee","Jung-Woo Ha","Jin-Hwa Kim","Sang-Woo Lee","Hwaran Lee","Hyun Oh Song"],"author_count":7,"year":2023,"published":"2023-05-27","venue":null,"cited_by":null,"doi":null,"arxiv_id":"2305.17444","nct_id":null,"source_url":"https://arxiv.org/abs/2305.17444","archive_url":"https://space-kitty.com/papers/arxiv:2305.17444","updated":"2023-05-27","pmid":null,"pmcid":null,"openalex_id":null,"pdf_url":"https://arxiv.org/pdf/2305.17444","abstract":"The deployment of large-scale generative models is often restricted by their potential risk of causing harm to users in unpredictable ways. We focus on the problem of black-box red teaming, where a red team generates test cases and interacts with the victim model to discover a diverse set of failures with limited query access. Existing red teaming methods construct test cases based on human supervision or language model (LM) and query all test cases in a brute-force manner without incorporating any information from past evaluations, resulting in a prohibitively large number of queries. To this end, we propose Bayesian red teaming (BRT), novel query-efficient black-box red teaming methods based on Bayesian optimization, which iteratively identify diverse positive test cases leading to model failures by utilizing the pre-defined user input pool and the past evaluations. Experimental results on various user input pools demonstrate that our method consistently finds a significantly larger number of diverse positive test cases under the limited query budget than the baseline methods. The source code is available at https://github.com/snu-mllab/Bayesian-Red-Teaming.","abstract_license":"CC0-1.0 (arXiv descriptive metadata)","license":null,"is_open_access":false,"retracted":false,"peer_reviewed":false,"categories":["cs.AI","cs.CL","cs.CR","cs.LG"],"details":null,"provenance":"arxiv","attribution":"Abstract: arXiv descriptive metadata (CC0). Thank you to arXiv for use of its open access interoperability. This service was not reviewed or approved by, nor does it necessarily express or reflect the policies or opinions of, arXiv.","topics":[],"cited_in_dossiers":[]}