Ratatouille
A Cross-Device Experience for Cooking
November 2024
My role
UX Research
Prototyping
Usability Testing
Team
Karoline Santos
01. The Challenge
Develop a transmedia experience centered on the user, integrating smartphone, smartwatch, and smart glasses (visionOS) in a seamless, context-aware, and device-specific way.
The challenge included creating a design brief for the three devices and preparing a 5-minute pitch to present a functional concept, rather than a finished product. We were encouraged to use AI tools to support contextual research and solution definition, focusing on accessibility, inclusivity, and identification of friction points.
This project was developed as part of the Programming Usable Interfaces module in the Carnegie Mellon Portugal program.
Objectives
- Build a cohesive, intuitive, and adaptable cross-device experience
- Leverage the strengths of each device to support real-time, contextual interactions
- Explore multimodal inputs: voice, gesture, touch, and haptic feedback
Core principals:
- Device-specific capabilities
- Context of use (home, mobility, public space)
- Cross-device continuity
Research Question
Does using multiple devices while cooking improve functionality, flow, and efficiency?
02. Research
2.1 Case studies
To answer our Research Question, we explored immersive and cross-device experiences such as:
- Nike Run Club – integration of app, wearable and real-time feedback
- HeartX / Gucci AVP – XR interfaces for healthcare and storytelling
Mixed reality cases using gesture and voice control
2.2 Proto persona
We created a proto persona to understand our possible users.
Meet Alice, a 26 years old, designer, who lives alone in Lisbon. She’s a beginner in the kitchen and wants clear, visual guidance to cook independently.
2.3 User Journey Map
We mapped Alice’s journey — from the moment she decides to cook to the moment she finishes the dish.
03. Ideation
We mapped key stages in the cooking process:
Before:
Explore, compare, and select recipes
During:
Receive clear guidance and real-time assistance
After:
Save, refine, and revisit preferred recipes
3.1 Sketching
3.2 Conceptual Metaphor: The "Mini Chef"
We drew inspiration from Remy, the mouse in Ratatouille, who acts as an invisible mentor. Similarly, our concept is a kind of digital sous-chef: quiet, helpful, always present.
- Vision Pro – the mentor: immersive, visual step-by-step
- Smartwatch – the timekeeper: alerts and tactile feedback
- Smartphone – the pantry: browse, choose, and personalize
The project was named after the movie Ratatouille with the goal of creating a sense of familiarity and reducing cognitive load
04. The Solution
4.2 Low-Fidelity Prototypes & Testing
As part of the experimental design process, we created paper prototypes for each device.
These were tested in a simulated kitchen environment to assess the feasibility of hands-free interactions.
What we learned:
- Visual overlays improved clarity and confidence
- Haptic alerts were intuitive and well-timed
- Switching between devices worked when transitions were minimal
- Testing in a simulated kitchen setting gave us a more realistic understanding of how users move, interact, and make decisions — including when they hesitate, need both hands, or stop to clean up
To improve:
- Visual alerts need more emphasis
- Break long steps into micro-interactions
- Include written alternatives for accessibility
4.3 Testing on Real Devices
We simulated basic interactions using a headset with basic gesture simulation
These tests were conducted with 4 participants, allowing us to gather varied feedback on the usability and comfort of cross-device interactions in a kitchen setting.
While not functional prototypes, these simulations helped validate interaction flow, timing, and the logic of device transitions.
However, participants raised two key concerns:
- Accessibility and perceived risk: Users were hesitant to use an expensive headset in a cooking environment, where steam, grease, or spills could pose damage risks. This feedback led us to refine the concept — the headset should be reserved for short, focused tasks, such as learning how to chop an onion, rather than guiding an entire recipe.
- Ergonomics: Some users reported that the weight of the device made it uncomfortable to wear for extended periods, reducing its practicality for long sessions.
These findings reminded us that good interaction design doesn’t just account for functionality — it must also consider emotional and physical comfort, as well as the perceived value and fragility of emerging technologies.
05. Impact and Future
Impact
- Reduced friction and cognitive load during cooking
- Enabled beginners to feel confident and supported
- Demonstrated potential for transmedia experiences in manual tasks
Next Steps
- Extend concept to new domains (e.g., medical training, arts and crafts)
- Build functional, sensor-enabled prototypes
- Conduct inclusive testing across diverse user groups
06. Personal Reflection
This project allowed us to test a concept, not deliver a final product. The pitch presented a vision of how transmedia, cross-device design can enhance embodied, hands-on learning.
It challenged us to think beyond screens — considering space, rhythm, gesture, and the physicality of interaction.
It also reminded us that when designing for emerging technologies, we must account for user perception, comfort, and context.
A headset may be immersive, but if it’s too heavy or too precious to risk in the kitchen, it changes how — and whether — people use it.
That’s why we believe immersive devices are best applied to short, focused micro-tasks where they provide the most value without getting in the way.