Hey hey! I'll be at #UIST2024 to present 𝗖𝗼𝗼𝗸𝗔𝗥 ☺️. CookAR is an AI-powered wearable AR prototype that augments kitchen tool affordances in near real-time for low vision users, enabling safer and more efficient tool interactions. (1/6)

Additionally, I want to extend a heartfelt thank you to all the researchers and recruiters I connected with during this year's internship search. I was fortunate to receive multiple amazing offers, and it was truly difficult to make a decision.

Andy's work in HCI and XR has been a huge inspiration since the beginning of my research journey, and collaborating with him has always been a dream of mine. I am beyond excited for this opportunity :)).

I'm thrilled to share that I'll be joining @MSFTResearch this summer, working with the incredible Andy Wilson!. I can’t wait to dive into the innovative projects ahead with Andy and the IMAIS team at MSR. Stay tuned, cause big things are coming! 😉.

RT @dj_hci: 📢📢 The Soundability Lab is recruiting a postdoc in the HCI & accessibility area. We’re also recruiting….

RT @_weimf: * interrupting your election doomscrolling *. I'm on the job market! I address online abuse as the next frontier of security &….

RT @da_lind: Finally, I am planning to recruit 1-2 PhD students in the coming cycle. If you are at #ISMAR2024 and interested in adaptive XR….

RT @vhshen: did anyone take this bag but with my wallet and keys inside? think somebody accidentally did a bag switcheroo - pls message me….

Huge thanks to my co-authors: Andrew Tjahjadi, Jiho Kim, Junpu Y., Minji Park, Jasmine Zhang, Jon Froehlich, Yapeng Tian, and Yuhang Zhao!. This work was awarded the 𝗕𝗲𝗹𝗼𝗻𝗴𝗶𝗻𝗴 & 𝗜𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻 𝗕𝗲𝘀𝘁 𝗣𝗮𝗽𝗲𝗿 𝗮𝘄𝗮𝗿𝗱 😄. See you soon! 🍳 👓 (6/6).

The study also revealed preferences for augmentation designs and highlighted five newly desired affordance areas: entry, exit, containment, intersection, and activation. (5/6).

In a user study with 10 low vision participants, we observed that users preferred affordance augmentations over traditional whole object augmentations during complex tasks like cooking. (4/6).

CookAR overlays green masks on grabbable areas (e.g., a knife handle) and red masks on hazardous areas (e.g., a knife blade). (3/6).

To build CookAR, we manually labeled an egocentric affordance dataset of cooking tools and fine-tuned an affordance segmentation model. The AR prototype integrates an Oculus Quest 2 VR headset with the ZED Mini stereo camera for computer vision and depth estimation. (2/6).

I'm traveling around Europe (London ➡️ Paris ➡️ Brussels ➡️ Amsterdam ➡️ Barcelona ➡️ Venice ➡️ Milan ➡️ Lucerne ➡️ Zurich), and will fly to Pitts :)).

See you back in Seattle! :)) (Or at UIST! :))).

The second CHI submission is from my previous internship at @RealityLabs with Tianyi Wang, Jacqui Fashimpaur, Naveen Sendhilnathan, and Tanya Jonker, PhD. I can't wait to share this one as well! I want to echo my post earlier in the year and say thank you again! :)).

And another huge thank you to my intern cohort: Nels Numan (@NelsNuman), Ziwei Liao, Ahmed Abdelreheem, and Bryan Tan! Ya'll are awesome, let's stay awesome 🤩. Thanks for hanging out with me outside work :)).

I learned a ton, and I can't wait to see what Niantic does next :)) Pokemon GO is extremely cool, but what this team is trying to accomplish is even more exciting 😉.

A huge thank you to my co first-author Filippo Aleotti, managers Jessica Van Brummelen & Gabriel Brostow, and the rest of the team (Diego Mazala, Guillermo Garcia-Hernando, Sara Vicente, Oliver Johnston, Isabel Kraus-Liang, and anyone else I forgot 😆).

As I wrap up two paper submissions to #chi2025, I want to say thanks to the brilliant team at @NianticLabs for making this internship extra awesome!. This crazy & amazing team made an awesome submission to CHI in less than 3 months, and I can't wait to share it!.