STADT:up Conclusion: What three years of UX research in autonomous driving have taught us

One of the central insights from our research: Vehicle occupants do not always want to know everything. Their need for information depends heavily on how safe or dangerous they perceive a situation to be. In critical moments — a confusing intersection, a pedestrian crossing, a sharp curve — the desire for concrete explanations increases significantly. What does the vehicle see? What is it planning? A well-designed iHMI provides information exactly when it is needed and holds back when it would be disruptive.

At the same time, we found that this need is highly individual. While some users prefer continuous, detailed feedback to feel safe, others quickly find the same density of information disruptive.

This insight has far-reaching consequences for the UX design of autonomous systems: A static interface is not enough. It requires adaptive, context-dependent, and personalizable interaction concepts upon which our overall iHMI concept is built.

In the second half of 2025, we conducted a user study in Berlin with 30 participants, which evaluated our overall iHMI concept in its entirety, including the newly developed components AR Windshield (Augmented Reality windshield) and voice announcements.

The result for the AR visualization was clear: When the vehicle shows in real-time what it perceives and how it assesses the situation, trust increases measurably. AR creates transparency — and transparency creates acceptance. The requirement for UX design is high: the visual cues must be intuitively understandable without being overwhelming. Too much information at the wrong moment quickly achieves the opposite.

The evaluation of the voice announcements showed a much more nuanced picture. Auditory signals can noticeably improve the user experience, but only if they are used in a targeted and contextual manner. For HMI design, this underlines an iron clad rule: Multimodality is not an end in itself, but must specifically serve understanding and the sense of safety.

One of the most significant methodological developments in the project was the transition of our simulation environment into Virtual Reality (VR). While we still evaluated the HMI via fixed screens as vehicle windows during the previous study in Berlin, we hit spatial and technological limits there when investigating continuous drives. Furthermore, in earlier phases, we mostly tested short, isolated scenarios of about 30 seconds – good for quick iterations of individual components, but limited in realistic expressiveness.

In the second half of 2025, we fundamentally renewed the simulation: A continuous, four-minute drive through real urban environments, completely immersively mapped in VR. In addition, the test subjects performed a cognitive secondary task (Non-Driving Related Task / NDRT) during the autonomous drive: a sequence of letters read aloud, where they had to react to specific target letters by raising their arm.

Why is this relevant for UX research?

Because trust in automation cannot be measured in short 30-second snippets. Trust is built through continuous experience of system behavior, through the sum of small moments in which the vehicle acts predictably, correctly, and understandably. The dual-task paradigm additionally provided us with two crucial insights:

1. Keeping an eye on the load limit: As long as the drive is smooth, occupants can also process detailed information well. However, when a complex traffic situation and personal distraction come together, many interfaces quickly lead to measurable stress.

2. The pitfall of the voice assistant: Our data showed that, above all, the combination of visual displays and simultaneous voice announcements distracted the test persons. When the car speaks and displays text in parallel, the occupants must constantly cross-reference both sources in their heads. This blocks mental capacities that were actually intended to be used for something else.

After the technical integration of the VR environment was successfully completed at the end of 2025, the final study was able to start on time in January 2026. The subsequent analysis of the data provided us with valuable insights for future HMI development, especially regarding cognitive load.

A clear limit to information density emerged: With a higher number of simultaneously active interfaces, the cognitive load of the test subjects increased measurably. When increased external stressors were added, conventional display concepts reached their limits.

The findings confirm our adaptive approach: More interfaces do not automatically mean more trust. The future of the iHMI lies in intelligent, situational orchestration that adapts both to the complexity of road traffic and to the individual preferences of the occupants. Only in this way can we create a user experience that builds trust and simultaneously relieves rather than overwhelms – thereby laying the foundation for true acceptance of autonomous driving.

While the passenger perspective dominates public discourse, in Subproject 5 we dedicated ourselves to an often overlooked user group: the developers, design researchers, and engineers who train, validate, and optimize AI algorithms for autonomous vehicles. Without their daily feedback, their field work in the test vehicle, and their data work, there is no learning system and thus no autonomous mobility. Until now, however, efficient development has often failed due to the so-called "workflow gap" – a massive gap between technical engineering and user-centered design.

For them, we developed the Engineering UI: a touch-optimized, web-based application that visualizes sensor data in real time, makes recordings structured and annotatable, and makes work in and on the test vehicle significantly more efficient. The technological foundation is the Robot Operating System (ROS2) and a strategic fork of the formerly open-source Foxglove code base, extended by custom nodes developed in-house for data compression for mobile use on tablets. The result: A significant reduction in cognitive load for operators during test drives.

Following the UX principle of "At-a-Glance Status Monitoring", the sensor status view was integrated from the navigation menu into the permanent sidebar. Faulty sensors are immediately identified by warning symbols, while bandwidth and frequency are monitored in real-time to proactively detect latency fluctuations. Seconds count in the vehicle — this UX detail has direct operational relevance for avoiding cognitive overload.

Finding relevant moments in hours of sensor data recordings was previously one of the biggest efficiency problems. Our further developed "Contextual Event Flagging" solves it: Instead of six categories, operators now only have to choose from three clearly differentiated flag types via touch during the drive: Red for critical situations, Yellow for warnings, White for information. In replay mode, events are visualized on an interactive timeline. The result: Searching large datasets is simplified and the analysis phase in post-hoc evaluation is more efficient.

The framework proves its scalability: It runs without further modifications both on the highly complex Mercedes-Benz S 580 research vehicle (15 sensors) and the compact research bicycle (FUSE-Bike) of the Munich University of Applied Sciences, as well as on the research vehicle from Opel. The underlying architecture is a dedicated proxy computer (Intel NUC) with specialized custom ROS2 nodes for data compression (downsizing), system monitoring, and bidirectional communication. To further increase performance in live operation, we also said goodbye to resource-intensive Docker containerization as an important learning and switched to a more stable native installation. This ensures an overall more stable application, making sensor recordings run more smoothly and live transmission take place with less latency.

A conceptual step forward that points far beyond the current project phase of STADT:up: In the second half of 2025, we validated and sharpened the complementary persona "Dara Designer" through contextual inquiries and synthesis workshops. She complements the established "Eggart Engineer" and aligns the strategic focus on the future: While the technical focus in the project was on the engineers, the Engineering UI is intended to serve not only them in the future, but also to allow UX designers access to interpreted live data directly in the test vehicle. This allows visualization concepts for end users to be evaluated under real driving conditions. This is the direct bridge between technical system development and human factors research.

The findings from STADT:up are finding their way directly into the international research community. Already in August 2025, the paper on the co-creative development of vehicle HMIs (Flohr et al., 2025) was published at the Humanist Conference in Chemnitz. A special milestone for our team: For HCI International 2026, the paper "Beyond the Dashboard: A Collaborative UI Framework for Autonomous Systems Engineering and Design Research" was not only submitted but also officially accepted — a direct further development of our work on the Engineering UI, together with research partners from the Munich University of Applied Sciences and Opel.

It was an intensive experiment in interdisciplinary collaboration at the highest level, between OEMs like Opel and Mercedes-Benz, Tier-1 suppliers, universities, and a UX agency that firmly believes: Technology only unfolds its full potential when it is made for people.

The insights we take with us can be brought down to a common denominator: Trust in autonomous systems is not only a technical problem, but also a design problem. It is not built through better algorithms alone, but through understandable, adaptive, and human-centered communication between human and machine. Exactly there lies the core competence of a UX agency like Ergosign.

We will further develop and apply the empirical insights and methods gained — from the VR simulation environment and adaptive iHMI concepts to the scalable Engineering UI framework — in future projects. The project's major final demonstration took place in Aldenhoven in June 2026.