Towards Provably Secure Generative AI: Reliable Consensus Sampling
Yu Cui Hang Fu Sicheng Pan Zhuoyu Sun Yifei Liu Yuhong Nie Bo Ran Baohan Huang Xufeng Zhang Haibin Zhang Cong Zuo Licheng Wang
作者信息
Abstract
Existing research on generative AI security is primarily driven by mutually reinforcing attack and defense methodologies grounded in empirical experience. This dynamic frequently gives rise to previously unknown attacks that can circumvent current detection and prevention. This necessitates the continual updating of security mechanisms. Constructing generative AI with provable security and theoretically controllable risk is therefore necessary. Consensus Sampling (CS) is a promising algorithm toward provably secure AI. It controls risk by leveraging overlap in model output probabilities. However, we find that CS relies on frequent abstention to avoid unsafe outputs, which reduces utility. Moreover, CS becomes highly vulnerable when unsafe models are maliciously manipulated. To address these issues, we propose a new primitive called Reliable Consensus Sampling (RCS), that traces acceptance probability to tolerate extreme adversarial behaviors, improving robustness. RCS also eliminates the need for abstention entirely. We further develop a feedback algorithm to continuously and dynamically enhance the safety of RCS. We provide theoretical guarantees that RCS maintains a controllable risk threshold. Extensive experiments show that RCS significantly improves robustness and utility while maintaining latency comparable to CS. We hope this work contributes to the development of provably secure generative AI.引用本文复制引用
Yu Cui,Hang Fu,Sicheng Pan,Zhuoyu Sun,Yifei Liu,Yuhong Nie,Bo Ran,Baohan Huang,Xufeng Zhang,Haibin Zhang,Cong Zuo,Licheng Wang.Towards Provably Secure Generative AI: Reliable Consensus Sampling[EB/OL].(2025-12-31)[2026-01-16].https://arxiv.org/abs/2512.24925.学科分类
计算技术、计算机技术
|国家预印本平台
评论