Developing the next generation of AI assistant

NYU Tandon researchers, led by Institute Professor Claudio Silva are exploring knowledge transfer, perceptual grounding, perceptual attention, and user modeling to develop dynamic, intelligent agents that can assist with complex tasks.

“The vision is that when someone is performing a certain operation, this intelligent agent would not only guide them through the procedural steps for the task at hand, but also be able to automatically track the process, and sense both what is happening in the environment, and the cognitive state of the user, while being as unobtrusive as possible. ... What we make is really designed for real-world applications, systems for people to use, rather than as theoretical exercises,” said Silva.