TIMI - Heads-up Context-aware Multimodal Travel Assistant [Design Portfolio]

🌠 TIMI - AR Travel Assistant Project

📝 Overview:
Many tourism-related mobile applications are frequently used by travelers, but the heads-down interaction they require can detract from the immersive experience of traveling. Our project, TIMI, is designed as a heads-up, context-aware multimodal travel assistant that aims to address these challenges. By leveraging AR glasses, TIMI enhances user experience across four key tasks: information retrieval, note-taking, photo-taking, and navigation. The design integrates these functions seamlessly into the travel experience, allowing users to digitally interact with their environment without losing the immersive joy of exploring new places.

🎨 Roles:
Lead Interaction Designer:
1️⃣ Directed user research through surveys, contextual inquiries, and think-aloud sessions, gathering qualitative data on user behavior when using smartphones for information retrieval etc. during tours.
2️⃣ Employed affinity diagrams to identify the main reasons, methods, and pain points in users' current digital interactions with their physical environment during tours to inform further design.
3️⃣ Created low-fidelity wireframes and developed a high-fidelity interactive prototype for the task of information retrieval for a context-aware travel assistant on AR smart glasses, using Figma.
4️⃣ Evaluated the high-fidelity prototype with real users in a lab environment using the Wizard-of-Oz method combined with gaze trackers.

✨ Extra Miles:
1️⃣ Non-traditional UI/UX Design: Persuaded my teammates and the course professor to initiate the first AR glasses UI/UX design portfolio in the course, advocating for more future-ready and creative designs that push the boundaries of traditional user interface design.
2️⃣ Figma for AR UI/UX Prototyping: Simulated the first-person view and eye gaze of the AR glasses user using Figma and mouse cursor, intuitively showcasing the user interface and interaction design via a traditional design platform. AR UI/UX prototyping is not taught as part of the course.
3️⃣ Comprehensive Literature Review: Conducted an in-depth review of literature on human-computer interaction, heads-up computing, AR glasses, etc., to justify design choices. AR and smartglasses related reading is not provided in the course.
4️⃣ Wizard-of-Oz AR User Evaluation: To quickly test and update the final interactive prototype, I used a pair of gaze trackers borrowed from the NUS-HCI Lab. This allowed me to track users' eye movements in real time and simulate their gaze movement using a mouse cursor on the high-fidelity prototype to simulate a functional AR glasses application through the Wizard-of-Oz method. None of this was taught in the course, and I had to source the equipment and space for testing.

💰 Results:
1️⃣ Full-Score Project: Led a multidisciplinary team of four NUS undergraduates from computer engineering, computer science, and industrial design backgrounds to achieve perfect scores in all three project milestones.
2️⃣ First AR Glasses UI/UX Design Portfolio in Course History: Produced the course's first AR smart-glasses design portfolio.
3️⃣ Invited Speaker to Future Cohorts: I was invited back to share this project with the next batch of CS3240 students, discussing AR glasses prototyping and how to approach UI/UX design in this emerging field.