We had a fascinating and informative experience in REO’s Biometrics Lab. The analysed data has led to valuable discoveries, which have been used to improve personalisation, reliability, and engagement for our human-AI experience.
Mercer
Our contributions
- Usability Testing
- Human-AI Experience
- Biometrics Research
- Heart Rate Monitoring
- Galvanic Skin Response
- Eye-Tracking
- Emotional Experience Analysis
- Experimentation Strategy
Challenges
To strengthen their respected brand, Mercer Master Trust decided to develop an AI pension agent capable of providing tailored information and guidance.
The business was already prioritising usability and satisfaction with a traditional UX interface, but this expansion sought to integrate a human-AI experience and create a more interactive, intuitive platform.
Mercer knew they had to ensure the advisor addressed real user needs for the project to succeed. Understanding that emotional design plays a crucial role in the human-AI experience, they set out to evaluate how users would perceive the tool’s trustworthiness and reliability while also gauging the emotional impact of interactions with the AI.
Mercer Master Trust approached REO Digital with three main objectives:
- Determine user interaction value with the AI agent to inform iterative improvement
- Obtain user sentiment data on the credibility and accuracy of the information received
- Measure the influence of unbiased emotional responses on user decisions
The REO approach
To de-risk this human-AI experience, REO evaluated user trust and value in Mercer’s AI pension agent by reaching beyond traditional UX research methods.
These methods can fall short in the world of human-AI interactions as these systems learn and adapt over time, changing behaviour at a rapid pace. They involve a complex emotional choreography in user interactions.
REO utilised more sophisticated and contemporary biometric responses such as heart rate, skin conductance, eye-tracking, and facial expression analysis. These tools offer real-time objective data on users’ emotional and cognitive states during interactions with the AI agent.
REO recruited 12 participants for 90-minute in-person testing sessions, ensuring that a variety of people were involved. Testers ranged from Mercer Master Trust members and new employees only just starting to think about their pensions, to people approaching retirement who are increasingly thinking about their pensions.
Tracking Biometrics
Participants performed a series of tasks using the AI agent designed to mimic realistic pension guidance scenarios.
This approach afforded REO a way to navigate the maze of self-report biases and probe each user’s behavioural interactions and emotional choreography throughout their experience.
The outcome
The advanced emotional biometric research conducted offered the Mercer Master Trust valuable, in-depth insights into how their members interacted with the AI pension agent.
These insights were highly actionable and revealed specific patterns in member behaviour, which could drive improvements. For example:
Personalisation Enhancement
Boosting Trust around Personal Data
Improved Conversational Accuracy
Enhanced Guidance Mechanisms
Increasing Engagement with Multimedia Features
Elevated User Experience
