Vpi-bench: Visual prompt injection attacks for computer-use agents

Analysis

Why This Paper Matters

Computer-use agents (CUAs) are an emerging class of AI systems that can interact with graphical user interfaces to perform tasks like web browsing, file management, and software operation. Their ability to access and manipulate sensitive data makes them attractive targets for malicious attacks. While prompt injection attacks have been studied in the context of text-based LLMs, visual prompt injection—where malicious instructions are embedded in images or screen content—poses a unique and under-explored threat to CUAs, which rely heavily on visual perception.

This paper introduces VPI-Bench, the first comprehensive benchmark specifically designed to evaluate visual prompt injection attacks against CUAs. By systematically categorizing attack vectors and goals, and by testing multiple state-of-the-art agents, the authors provide a critical resource for understanding and mitigating these security risks. The benchmark's findings are alarming: all tested agents are vulnerable to visual prompt injection, often with high success rates. This work is essential for the AI community as it underscores the urgent need for robust defenses in autonomous systems.

Technical Contributions

• Benchmark Design: VPI-Bench includes a diverse set of tasks (e.g., email management, online shopping, coding) and a variety of visual prompt injection attacks, such as hidden text, overlays, and adversarial images.
• Attack Taxonomy: The paper categorizes attacks based on the injection vector (e.g., webpage content, pop-up dialogs, images) and the attacker's goal (e.g., data exfiltration, task hijacking, denial of service).
• Evaluation Framework: The authors propose metrics for both task completion (benign performance) and attack success rate (malicious performance), enabling a nuanced assessment of agent security.
• Empirical Study: They evaluate several commercial and open-source CUAs, including those based on GPT-4V, Claude, and LLaVA, providing a comparative analysis of their vulnerabilities.

Results

The paper reports that all evaluated CUAs are susceptible to visual prompt injection attacks, with attack success rates ranging from 20% to over 80% depending on the attack type and agent. For instance, simple text overlays on webpages achieved high success rates across all agents, while more subtle attacks (e.g., small text in images) were less effective but still dangerous. The study also found that attack success is influenced by the size and placement of the injected text, with larger and more central text being more likely to be followed. Additionally, the authors observed that some agents are more robust than others, but none are immune, highlighting the general nature of the vulnerability.

Significance

This paper has significant implications for the safe deployment of computer-use agents. By providing a standardized benchmark, it enables researchers and developers to systematically evaluate and improve the security of their systems. The findings serve as a wake-up call for the AI community, emphasizing that visual prompt injection is a real and present threat that must be addressed before CUAs are widely adopted. The benchmark also opens up new research directions, such as developing defense mechanisms like input sanitization, instruction hierarchy, and anomaly detection. Ultimately, this work contributes to the broader goal of building trustworthy AI agents that can operate autonomously without compromising user security and privacy.