morelikely to bypass safety filters than a query that is openly harmful.
The V-DEAL Diagnosis #
The V-DEAL framework breaks down this vulnerability across three levels: behavior, understanding, and internal representations. The data from six different Video LLMs reveals a weird contradiction:
Recognition Accuracy: Over 81% (the models know exactly what they are looking at).Attack Success Rate: 48.33% (they still provide the harmful output when the query is benign).
When looking at the hidden states, the internal "refusal tendency" is significantly weaker when triggered by visual understanding compared to textual understanding. Essentially, the model's "safety alarm" is tuned to listen to the text, but it's nearly deaf to the visuals.
Fixing the Gap #
Instead of relying on expensive fine-tuning, a prompt injection intervention can bridge this gap. By forcing the model to better couple its visual perception with its refusal mechanism, the attack success rate dropped by an average of 48.24 percentage points.
This suggests that the "safety" of Video LLMs isn't about the model lacking the ability to see danger—it's about a failure in the internal wiring that connects "I see something bad" to "I should not answer this." This is a critical insight for anyone working on an AI workflow involving multimodal inputs, as it proves that sanitizing the text prompt isn't enough if the video content itself is the vector.
Next Claude Prompt Injection: When the AI steers the User →