Large vision-language models (LVLMs) are vulnerable to typographic attacks—misleading text within images that overrides visual understanding. We introduce Read-or-Ignore VQA (RIO-VQA) and its benchmark RIO-Bench, a unified evaluation framework …
We propose Text-Printed Image (TPI), which generates synthetic images by directly rendering textual descriptions on a plain white canvas, bridging the modality gap between text and images for cost-efficient training of large vision-language models. …
We investigate how Differential Attention (DA) affects adversarial vulnerability in vision transformers and CLIP models. While DA suppresses redundant context via a subtractive structure, we show it paradoxically increases adversarial sensitivity …
Vision-Language Models (VLMs) are increasingly adopted in practical applications, but remain vulnerable to adversarial perturbations. Existing adversarial fine-tuning methods often rely on one-to-one image-text supervision and may overfit to narrow …
We propose PAChroma, a method that embeds imperceptible perturbations into grayscale images to prevent unauthorized AI colorization. The resulting Uncolorable Examples resist AI colorization while maintaining visual quality, transferability, and …
Defending pre-trained vision-language models (VLMs), such as CLIP, against adversarial attacks is crucial, as these models are widely used in diverse zero-shot tasks, including image classification. However, existing adversarial training (AT) methods …
Protecting the intellectual property of Large Language Models (LLMs) has become increasingly critical due to the high cost of training. Model merging, which integrates multiple expert models into a single multi-task model, introduces a novel risk of …
Although adversarial training has been the state-of-the-art approach to defend against adversarial examples (AEs), it suffers from a robustness-accuracy trade-off, where high robustness is achieved at the cost of clean accuracy. In this work, we …
Infrared detection is an emerging technique for safety-critical tasks owing to its remarkable anti-interference capability. However, recent studies have revealed that it is vulnerable to physically-realizable adversarial patches, posing risks in its …
Calibrating deep learning models to yield uncertainty-aware predictions is crucial as deep neural networks get increasingly deployed in safety-critical applications. While existing post-hoc calibration methods achieve impressive results on in-domain …