Situated Interaction
| Dan Bohus (Microsoft Research, Redmond, Washington, US) |
|---|
Abstract
Physically situated dialog is a complex, multimodal affair that goes well beyond the spoken word. When interacting with each other, people incrementally coordinate their actions to simultaneously resolve several different problems: they manage engagement, coordinate on taking turns, recognize intentions, and establish and maintain common ground as a basis for contributing to the conversation. A wide array of non-verbal signals are brought to bear. Proximity and body pose, attention and gaze, head nods and hand gestures, prosody and facial expressions, all play important roles in the intricate, mixed-initiative, fluidly coordinated process we call interaction. And just like a couple of decades ago advances in speech recognition opened up the field of spoken dialog systems, today advances in vision and other perceptual technologies are again opening up new horizons -- we are starting to be able to build machines that can understand these social signals and the physical world around them, and begin to participate in physically situated interactions and collaborations with people.
In this talk, using a number of research vignettes from my work, I will draw attention to some of the challenges and opportunities that lie ahead of us in this exciting space. In particular, I will discuss issues with managing engagement and turn-taking in multiparty open-world settings, and more generally highlight the importance of timing and fine-grained coordination in situated interaction. Finally, I will conclude by describing a framework that promises to simplify the development of physically situated interactive systems and enable more research and faster progress in this area.
Biography
Dan Bohus is a Senior Principal Researcher in the Perception and Interaction Group at Microsoft Research. His work centers on the study and development of computational models for physically situated spoken language interaction and collaboration. The long term question that shapes his research agenda is how can we enable interactive systems to reason more deeply about their surroundings and seamlessly participate in open-world, multiparty dialog and collaboration with people? Prior to joining Microsoft Research, Dan obtained his Ph.D. from Carnegie Mellon University.
Physically situated dialog is a complex, multimodal affair that goes well beyond the spoken word. When interacting with each other, people incrementally coordinate their actions to simultaneously resolve several different problems: they manage engagement, coordinate on taking turns, recognize intentions, and establish and maintain common ground as a basis for contributing to the conversation. A wide array of non-verbal signals are brought to bear. Proximity and body pose, attention and gaze, head nods and hand gestures, prosody and facial expressions, all play important roles in the intricate, mixed-initiative, fluidly coordinated process we call interaction. And just like a couple of decades ago advances in speech recognition opened up the field of spoken dialog systems, today advances in vision and other perceptual technologies are again opening up new horizons -- we are starting to be able to build machines that can understand these social signals and the physical world around them, and begin to participate in physically situated interactions and collaborations with people.
In this talk, using a number of research vignettes from my work, I will draw attention to some of the challenges and opportunities that lie ahead of us in this exciting space. In particular, I will discuss issues with managing engagement and turn-taking in multiparty open-world settings, and more generally highlight the importance of timing and fine-grained coordination in situated interaction. Finally, I will conclude by describing a framework that promises to simplify the development of physically situated interactive systems and enable more research and faster progress in this area.
Biography
Dan Bohus is a Senior Principal Researcher in the Perception and Interaction Group at Microsoft Research. His work centers on the study and development of computational models for physically situated spoken language interaction and collaboration. The long term question that shapes his research agenda is how can we enable interactive systems to reason more deeply about their surroundings and seamlessly participate in open-world, multiparty dialog and collaboration with people? Prior to joining Microsoft Research, Dan obtained his Ph.D. from Carnegie Mellon University.
Show speech transcript
