Investigating the role of conversational agents in augmented cultural heritage tours

Recent technological advancements in generative Artificial Intelligence (AI) models combined with Augmented Reality (AR) systems represent a new opportunity for cultural heritage valorization, particularly in the context of increasingly large, heterogeneous, and multimodal digital cultural repositories that pose challenges in terms of scalable access and semantic retrieval. In this article, we introduce ARtour, an application for cultural tours implementing a navigation system to direct users toward a point of interest and provide information through an interactive Large Language Model (LLM)-based audio system. Beyond enhancing user experience, the system explores the role of LLMs as conversational interfaces for accessing structured and unstructured cultural heritage data within digital twin environments, while also examining how such conversational interaction influences users’ experience, including usability, cognitive workload, and affective responses during cultural heritage exploration. A user study comparing LLM-based interaction with traditional web search shows that the system achieves high usability and technology acceptance with low cognitive workload. While task performance remains comparable across conditions, the AI conversational agent enhances user engagement and supports a more exploratory information-seeking behavior, increasing perceived immersion and co-presence and providing insights into the role of conversational agents in human–machine interaction within multimodal AR environments. We finally argue that the integration of fine-tuned or retrieval-augmented models for accessing information on historical and cultural artifacts could have a constructive impact on the promotion of publicly accessible cultural heritage, while raising important considerations regarding data accuracy, provenance, and governance.