Students tackle problems head-on in human-computer interaction course

In Fall 2024, Dishita Turakhia joined Tandon as a Faculty Fellow in the Department of Computer Science and Engineering with a clear goal in mind: to teach students to create technology that will enhance the human experience.  

She asked students in her foundational course in human-computer interaction (HCI) to choose problems they felt strongly about; apply HCI research methods such as user interviews, in-depth analyses, and design development; and come up with possible solutions, working to create either digital, virtual, or physical prototypes that highlighted both their creativity and technical skills. 

Students poured their efforts wholeheartedly into their projects, with some jaw-dropping results that were showcased at the Clive Davis Gallery at the end of the semester in a open-to-public demo exhibition attended by dozens of attendees. 

Take Ashley Sun, for example. With an undergraduate degree in economics, she entered Tandon with no experience in computer science or electronics, but by the time of the showcase, she had used a game joystick, sensors, and actuators to build a smart makeup brush capable of mimicking strokes on the face, with an accompanying digital app that applies cosmetics to a user’s virtual avatar. “I never expected to learn so much in a single semester,” she admits, “but I’m excited to move forward now and see if I can refine my prototype further. (Another student, Bonnie Lu, who is studying Computer Science and Business and Technology Management, also delved into the beauty world, tackling the problem of too-frequent retail returns by developing a virtual product tester.)

A significant number of other students tackled mental well-being, nutrition, and health, including:

• Michael Bian and Krish Murjani, who developed an app that uses augmented reality to help at-risk individuals identify symptoms of stroke, run diagnostic tests using their cellphone cameras, and automatically call 911 if needed. (During the tests, they are asked to do simple but revealing tasks such as raise their arms and read a line of text.)
• Srishti Arora and Shresth Kapoor, who developed an app capable of giving solid nutrition advice. (Simply let the app know if you have any underlying conditions like diabetes or any goals such as weight loss, and point your camera at a food product. You’ll receive feedback on whether or not the item is suitable for you – and if you decide to eat it anyway, how much physical activity would be needed to counteract the detrimental effects.)
• Zach Fan and Abby Santos, the creators of Pepper, an app that analyzes your nutritional intake based on a simple system of color-coding and the advice to eat a rainbow of fruits and vegetables each day. (Missing enough green vegetables or red berries? The app will let you know and suggest recipes.)
•  Sharayu Rasal and Tyler Song, who created an entire virtual ecosystem aimed at helping users prioritize their well-being, gain a sense of calm, and determine when they should speak to a trained human therapist. 

Others focused on financial health, such as Dan Zhang and Omar Shalaby, whose app combs through digital files to pinpoint which will be needed for tax prep and then provides personalized filing advice; and Penny Kasinphila and Jay Desai, whose AI-based Chrome extension helps inexperienced renters identify potentially fake listings on popular real estate sites like Zillow. 

It’s natural for students to be concerned with learning-related issues, and a handful of projects tackled ways to make educational content more accessible in rural India, incorporating an AI-based agent that provides personalized tutoring (Ishaan Mishra and Anurup Naskar) and to get students more organized, thanks to a smart calendar planning tool called Huli that simplifies planning for studying and projects based on the individual’s workload and stress levels (Helen Yuan and Jia Huang). And it’s just as natural for fun and games to be a focus, as evidenced by Tanishi Mishra and Ishita Paliwal’s haptic glove, aimed at aiding gamers with vision or hearing impairments; and Anitej Srivastava and Ricardo Cortes’s computer vision-based method of annotating live tennis games so that players and spectators can later analyze and learn from them.    

It was wonderful to see projects varying so widely. Many of them integrated technical skills learned in other courses, such as Computer Vision and Machine Learning, while also developing new skills like electronics prototyping. And whether they were focused on addressing everyday needs or making a broader social impact, I think they all learned the importance of taking the end user into consideration and engaging in human-centered design.” 
— Dishita Turakhia, Faculty Fellow

 

Special thanks to David Lasala, Director of NYU’s Immersive Technology Lab (iTLAB) and Liz New, MakerSpace manager for their enthusiasm and support.