Uncovered key customer usage patterns through on-site data analysis, directly informing iterative product improvements.
UX Research
Usability Testing
Company
Zalando
Project Duration
Mar 2024
Role
UX Researcher
Overview
The Zalando AI-Assistant team needed a systematic way to understand user behavior and identify failure points that were causing drop-offs and a poor user experience. I was brought in to establish a clear research framework that could continuously uncover usage patterns and provide a feedback loop for ongoing product improvements.
Research Approach
I led a human-in-the-loop analysis to systematically evaluate user interactions and identify opportunities for improvement. My approach was two-fold:
I conducted a
thorough analysis of 200 onsiteuser conversations with the AI assistant,focusing on identifying user intentand understanding key failure points that were hampering the user experience.
I analyzed our
internal LLMtool's capacity to correctly identify failures and tag assistant responses. This provided a crucial feedback loop to improve machine evaluations andbuild a more scalable analysisprocess for the future.
Impact
My research had a direct and lasting impact on the team's process and product. I successfully:
Established a continuous research formatto support our weekly annotation meetings. This provided a structured way to understand problem areas and identify clear improvement pointers.Delivered improvements to our internal annotation tools, which directly enhanced the machine evaluations of user conversations.
Provided the team with a clear,
data-driven understanding of user intents and needs,which became the foundation for continuous iteration and improvement of the Zalando Assistant.
Team
1 x Design Manager
1 x Engineering Manager
1 x Product Designer
3 x Applied Scientist
1 x User Researcher (me)
