When External Threats Become User Context: Tricking Agents into Accepting Malicious Information

Autonomous LLM agents receive task goals through user requests, but these requests may also relay untrusted external information. We study source relocation, where an externally available artifact is additionally included in a benign user's initial request while the task environment remains unchanged. Through controlled experiments across eight models and four AgentDojo domains, we find that user-context placement increases adoption of both malicious and benefit-matched benign artifacts, including without explicit artifact-use instructions. For GPT-4o in Travel, malicious-artifact adoption rises from 27.8\% to 95.6\%. Matched-checkpoint analyses of open-weight models reveal model-dependent differences in conditional artifact support, and activation patching identifies components contributing to this controlled score. The evaluated defense configurations leave substantial malicious execution under user-context placement. Explicit verification guidance raises GPT-4o's safe-completion rate from 2.2\% to 15.0\%,

Source Relocation; User-relayed Context Manipulation; LLM-based Agents; Agent Security; Human Factors in Cybersecurity; Web-Use Agents; Planning Agents

PDF

Use of a P3P user agent by early adopters

The Platform for Privacy Preferences (P3P), developed by the World Wide Web Consortium (W3C), provides a standard computer-readable format for privacy policies and a protocol that enables web browsers to read and process privacy policies automatically. P3P enables machine-readable privacy policies that can be retrieved automatically by web browsers and other user agent tools that can display symbols, prompt users, or take other appropriate actions. We developed the AT&T Privacy Bird as a P3P user agent that can compare P3P policies against a user's privacy preferences. Since P3P was adopted as a W3C recommendation in April 2002, little work has been done to study how it is being used and, especially, its impact on users. Many questions have been raised about whether and how Internet users will make use of P3P, and how to build P3P user agents that will prove most useful to end users. In this paper we first provide a brief introduction to P3P and the AT&T Privacy Bird. Then we discuss a survey of AT&T Privacy

125 citations

HTMLAltmetric

Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysis

Agent applications are increasingly adopted to automate workflows across diverse tasks. However, due to the heterogeneous domains they operate in, it is challenging to create a scalable evaluation framework. Prior works each employ their own methods to determine task success, such as database lookups, regex match, etc., adding complexity to the development of a unified agent evaluation approach. Moreover, they do not systematically account for the user's role nor expertise in the interaction, providing incomplete insights into the agent's performance. We argue that effective agent evaluation goes beyond correctness alone, incorporating conversation quality, efficiency and systematic diagnosis of agent errors. To address this, we introduce the TED framework (Talk, Evaluate, Diagnose). (1) Talk: We leverage reusable, generic expert and non-expert user persona templates for user-agent interaction. (2) Evaluate: We adapt existing datasets by representing subgoals-such as tool signatures, and responses-as natural

Accepted as a conference paper at ICLR 2026. Code and dataset are available in the repository https://github.com/SAP-samples/agent-quality-inspect

PDF

Personalisation for user agents

This paper is concerned with personalisation of user agents by symbolic, on-line machine learning techniques. The application of these ideas to an infotainment agent is discussed in detail. Also experimental results, which indicate that a high level of personalisation can be achieved by this approach, are presented.

19 citations

HTMLAltmetric

Beyond Task Completion: Measuring Interaction Cost in Terminal User Interfaces

Large language models (LLMs) are increasingly used through terminal user interfaces (TUIs), yet task completion alone does not capture how difficult an interface is to understand and operate. Existing human assessments and LLM-generated ratings or reports do not provide repeatable measurements of interaction effort grounded in verified task execution. We propose Agent-as-a-User, an evaluation paradigm that places an LLM agent in the user role. Agent-KLM separates agent-side interpretation and operation costs and relates them structurally to human interaction. TUINaut operationalizes the paradigm by recording interaction trajectories and verifying outcomes with actor-independent oracles. Using TUINaut-Bench, we study 84 tasks across 15 real-world TUIs with six human participants and five observation-evaluator configurations, and evaluate 45 TUIs generated by three leading stacks. Similar aggregate success rates mask differences in which tasks humans and agents complete, whereas interpretation and operation cos

Web Page: https://tuinaut.pages.dev/ Source Code: https://github.com/kinesiatricssxilm14/Agent-as-a-User

PDF

User Agents

In this chapter, we’ll discuss the specifics of user agents, which are user background processes.

0 citations

HTMLAltmetric

Evaluating Social Acceptance of eXtended Reality (XR) Agent Technology: A User Study (Extended Version)

In this paper, we present the findings of a user study that evaluated the social acceptance of eXtended Reality (XR) agent technology, focusing on a remotely accessible, web-based XR training system developed for journalists. This system involves user interaction with a virtual avatar, enabled by a modular toolkit. The interactions are designed to provide tailored training for journalists in digital-remote settings, especially for sensitive or dangerous scenarios, without requiring specialized end-user equipment like headsets. Our research adapts and extends the Almere model, representing social acceptance through existing attributes such as perceived ease of use and perceived usefulness, along with added ones like dependability and security in the user-agent interaction. The XR agent was tested through a controlled experiment in a real-world setting, with data collected on users' perceptions. Our findings, based on quantitative and qualitative measurements involving questionnaires, contribute to the understa

26 pages (18 pages main body, 8 pages user consent form), 3 figures, 7 tables

PDF

Adapting Web Content to Mobile User Agents

In this article, we describe an adaptation proxy we developed as a part of a Kontti research project at VTT Information Technology that lets mobile users access Web content that's not directly targeted to mobile user agents. More and more content is now available on the Internet, and there's a growing need for mobile users to be able to access it. Thus the authors describe the adaptation proxy, which lets mobile users access Web content that's not directly targeted to user agents of mobile devices. The adaptation proxy can adapt Extensible Hypertext Markup Language (XHTML) documents into XHTML mobile profile (XHTML MP) and Wireless Markup Language (WML), and can perform media adaptation. At the system's core is an adaptation framework to which new source and target XML languages can be introduced with relatively little effort.

89 citations

HTMLAltmetric

Influence of Agent Number and Behavior on Users' Psychological Responses in VR Evacuation Training

This study aims to build effective drill software by combining a virtual simulation of a disaster with multiple agents capable of expressing emotions. We evaluated how the number of agents and their emotional and behavioral actions during evacuation influence users in a virtual space. The results showed that users tended to act more confidently when many agents evacuated with them, and that the evacuation route chosen by the agents influenced the users' decision-making. These findings indicate that users' evacuation behavior is shaped not only by the environment itself but also by the collective behavior of surrounding agents, underscoring the need to consider how agent behavior affects users' psychological states during training.

The 2nd International Workshop on Trust and Blame in Multi-Person Multi-Agent Societies (TBSA_2nd) in conjunction with HAI 2026, 7 pages, 8 figures, 5 tables

PDF

Vulnerability Analysis of MMS User Agents

The Multimedia Messaging Service (MMS) is becoming more popular, as mobile phones integrate audio and video recording functionality. Multimedia messages are delivered to users through a multi-step process, whose end-points are the MMS User Agents that reside on the users mobile phones. The security of these components is critical, because they might have access to private information and, if compromised, could be leveraged to spread an MMS-based worm. Unfortunately, the vulnerability analysis of these components is made more difficult by the fact that they are mostly closed-source and the testing has to be performed through the mobile phone network, which makes the testing time-consuming and costly. This paper presents a novel approach to the security testing of MMS User Agents. Our approach takes into account the effects of the infrastructure on the delivery of MMS messages and then uses a virtual infrastructure to speed up the testing process. Our testing approach was able to identify a number of previously

39 citations

HTMLAltmetric

SWE-DIVE: Multi-Turn Coding Agent Evaluation with Dynamic User Simulation

Coding agents are typically evaluated using one-shot, well-formulated prompts. Benchmarks such as SWE-bench use existing GitHub issues as input for a coding agent and evaluate whether the agent successfully solves the task. However, in real-world use cases, prompts are often not as well-formulated or structured as existing GitHub issues. Additionally, a coding agent session is not limited to a single prompt; it typically involves a multi-turn interaction in which the user iteratively adds new tasks and requirements that the agent must fulfill. To benchmark these abilities, we developed SWE-DIVE, a dynamic, interactive validation and evaluation framework for benchmarking coding agents in a multi-turn environment. To achieve this, we introduce a user simulator that can simulate different behavioral patterns and communicate with the coding agent. We also introduce a conversation validator that evaluates the user simulator's outputs to determine whether they fulfill the initial requirements and guide the agent to

Accepted at the NeurIPS 2026 workshop "Grounded User Simulation for Model Evaluation and Training: Diversity, Fidelity, and Validity'' (non-archival)

PDF

User Agents Dataset

A collection of user agents in CSV, Excel and plain text file formats.

0 citations

PDFHTMLAltmetric

Toward User Comprehension Supports for LLM Agent Skill Specifications

Users often interpret and select agent skills through their SKILL markdown specifications. To protect users, existing audits mainly focus on malicious or unsafe skills. We study the complementary question of whether specifications help users form bounded expectations about what a skill consumes, produces, and covers. Across 878 cybersecurity skills, we used rule-based coding to measure textual cues for four comprehension anchors, namely operational basis, output contract, boundary disclosure, and example capability demonstration. Cues for operational basis were common, but only 19.0% of specifications exhibited cues for an example task, sample, or expected outcome, and only 2.3% exhibited cues for all four anchors. We further examined a small DNS/C2 telemetry subset (n$=$6) to illustrate why missing examples may matter. Examples appeared to make first local checks easier to construct, while no-example skills typically required helper code inspection to recover command arguments or output fields. We argue that

To appear at ACM CAIS Workshop Agent Skill 2026

PDF

Indicating User Agent Capabilities in the Session Initiation Protocol (SIP)

This specification defines mechanisms by which a Session Initiation Protocol (SIP) user agent can convey its capabilities and characteristics to other user agents and to the registrar for its domain. This information is conveyed as parameters of the Contact header field. [STANDARDS-TRACK]

90 citations

HTMLAltmetric

MRMMIA: Membership Inference Attacks on Memory in Chat Agents

Membership inference attacks (MIAs) test whether a target data record belongs to a system's private data, and have become a standard tool to measure privacy leakage in machine learning systems. Prior work has primarily focused on training corpora or retrieval databases. However, MIAs against agent memory have received less attention, even though such memory can contain sensitive user-agent interactions, retrieved facts, and user preferences. Therefore, in this work, we focus on chat agent memory MIAs, where an adversary infers whether a candidate memory unit belongs to the chat agent's memory store. We propose Multi-Recall Memory MIA (MRMMIA), a unified attack that utilizes multiple recall probes to the agent to extract the membership signal across black-box, gray-box, and white-box settings. Our experiments demonstrate that MRMMIA consistently outperforms baselines. Our results expose the privacy risk in agents and provide an initial evaluation framework for membership leakage in chat-agent memory systems.

This work investigates the MIA on chat agent memory

PDF

The Fiduciary Duties of User Agents

4 citations

PDFHTMLAltmetric

Delegating or Doing? Understanding User Behavior in Hybrid Human-Agent Interfaces

Large Language Models (LLMs) are increasingly embedded into applications, allowing users to complete tasks either through direct manipulation or by delegating actions to conversational agents. However, little is known about how users balance these modalities when both are available. We present a web-based content management system augmented with an LLM agent through the Model Context Protocol (MCP), enabling users to perform CRUD tasks through a graphical interface, a conversational agent, or both. We conducted a between-subjects study (N=73) comparing three interaction modes: Traditional-Only, AI-First, and Hybrid. Across sixteen scenarios, we analyzed task completion time, interaction logs, and delegation behavior. AI-assisted interaction significantly reduced clicks, page navigations, and scrolling indicating lower interaction effort. Surprisingly, these reductions did not translate into faster task completion, as task duration did not differ significantly across conditions. We also found no significant re

9 pages, In Proceedings of the 14th International Conference on Human-Agent Interaction at Osaka, Japan on November 16-19, 2026

PDF

On the structure and characteristics of user agent string

User agent (UA) strings transmitted during HTTP transactions convey client system configuration details to ensure that content returned by a server is appropriate for the requesting host. As such, analysis of UA strings and their structure offers a unique perspective on active client systems in the Internet and when tracked longitudinally, offers a perspective on the nature of system and configuration dynamics. In this paper, we describe our study of UA string characteristics. Our work is based on analyzing a unique corpus of over 1B UA strings collected over a period of 2 years by comScore. We begin by analyzing the general characteristics of UA strings, focusing on the most prevalent strings and dynamic behaviors. We identify the top 10 most popular User Agents, which account for 26% of total daily volume. These strings describe the expected instances of popular platforms such as Microsoft, Apple and Google. We then report on the characteristics of low-volume UA strings, which has important implications for

20 citations

HTMLAltmetric

Exploring the Impact of Non-Verbal Virtual Agent Behavior on User Engagement in Argumentative Dialogues

Engaging in discussions that involve diverse perspectives and exchanging arguments on a controversial issue is a natural way for humans to form opinions. In this process, the way arguments are presented plays a crucial role in determining how engaged users are, whether the interaction takes place solely among humans or within human-agent teams. This is of great importance as user engagement plays a crucial role in determining the success or failure of cooperative argumentative discussions. One main goal is to maintain the user's motivation to participate in a reflective opinion-building process, even when addressing contradicting viewpoints. This work investigates how non-verbal agent behavior, specifically co-speech gestures, influences the user's engagement and interest during an ongoing argumentative interaction. The results of a laboratory study conducted with 56 participants demonstrate that the agent's co-speech gestures have a substantial impact on user engagement and interest and the overall perceptio

9 pages, accepted to International Conference on Human-Agent Interaction (HAI '24), November 24--27, 2024, Swansea, United Kingdom

PDFAltmetric

Unified User-Agent String

User-Agent is a HTTP request-header field. It contains information about the user agent originating the request, which is often used by servers to help identify the scope of reported interoperability problems, to work around or tailor responses to avoid particular user agent limitations, and for analytics regarding browser or operating system use. Over the years contents of this field got complicated and ambiguous. That was the reaction for sending altered version of websites to web browsers other than popular ones. During the development of the WWW, authors of the new web browsers used to construct User-Agent strings similar to Netscape's one. Nowadays contents of the User-Agent field are much longer than 15 years ago. This Memo proposes the Uniform User-Agent String as a way to simplify the User-Agent field contents, while maintaining the previous possibility of their use.

0 citations

HTML