ASRU 2011

The expression and experience of human behavior are complex and multimodal, and are characterized by individual and contextual heterogeneity and variability. Speech and spoken language communication cues offer important means into measuring and modeling human behavior. In fact, observational research and practice across a variety of domains from commerce to healthcare rely on speech and language based informatics. Consider for example the domain of Autism where crucial diagnostic information comes from audiovisual data of verbal and nonverbal behavior. Similar reliance on observed interactions is common across therapeutic settings in mental health. Behavioral signal processing advances can enable not only new possibilities for gathering data in a variety of settings--from laboratory and clinics to free living conditions-but promise computational techniques and models to advance evidence-driven theory and practice.

This talk will describe some ongoing efforts on Behavioral Signal Processing-technology and algorithms for quantitatively and objectively understanding typical, atypical and distressed human behavior- with a specific focus on communicative, affective and social behavior. Using examples drawn from different domains, the talk will illustrate Behavioral Informatics applications of these processing techniques that contribute to quantifying higher-level, often subjectively described, human behavior in a domain-sensitive fashion. In particular, we will draw on examples from work on health domains to illustrate the challenges and opportunities for behavioral speech and spoken signal processing. [Work supported by NIH, DARPA, ONR, and NSF].