The Many Facets of Dialog
| Helen Meng (Chinese University of Hong Kong, China) |
|---|
Abstract
Dialog is a most fascinating form of human communication. The back-and-forth exchanges convey the speaker's message to the listener, and the listener can derive information about the speaker's thoughts, intent, well-being, emotions and much more. This talk presents an overview of dialog research that concerns our group at The Chinese University of Hong Kong. In the domain of education and learning, we have been recording in-class student group discussions in the flipped-classroom setting of a freshman elite mathematics course. We investigate features in the weekly, within-group dialogs that may relate to class performance and learning efficacy. In the domain of e-commerce, we are developing dialog models based on approximately 20 million conversation turns, to support a virtual shopping assistant in customer inquiries and orders, logistics tracking, etc. In the domain of health and wellbeing, we are capturing and analysing dialogs between health professionals (or their virtual equivalent) and subjects in cognitive screening tests. We also conduct research in both semantic interpretation and dialog state tracking, as well as affective design of virtual conversational assistants. For the former, we have developed a Convex Polytopic Model for extracting a knowledge representation from user inputs in dialog turns by generating a compact convex polytope to enclose all the data points projected to a latent semantic space. The polytope vertices represent extracted semantic concepts. Each user input can then be "interpreted" as a sequence of polytope vertices which represent the user's goals and dialog states. For the latter, we have developed a multimodal, multi-task, deep learning framework to infer the user's emotive state and emotive state change simultaneously. This enables virtual conversational assistants to understand the emotive state in the user's input and to generate an appropriate emotive system response in the dialog turn, which will further influence the user's emotive state in the subsequent dialog turn. Such an affective design will be able to enhance user experience in conversational dialogs with intelligent virtual assistants.
Biography
Helen Meng is Patrick Huen Wing Ming Professor of Systems Engineering and Engineering Management at The Chinese University of Hong Kong (CUHK). She is the Founding Director of the CUHK Ministry of Education (MoE)-Microsoft Key Laboratory for Human-Centric Computing and Interface Technologies (since 2005), Tsinghua-CUHK Joint Research Center for Media Sciences, Technologies and Systems (since 2006), and Co-Director of the Stanley Ho Big Data Decision Analytics Research Center (since 2013). Previously, she served as CUHK Faculty of Engineering's Associate Dean (Research), Chairman of the Department of Systems Engineering and Engineering Management, Editor-in-Chief of the IEEE Transactions on Audio, Speech and Language Processing, Member of the IEEE Signal Processing Society Board of Governors, ISCA Board Member and presently Member of the ISCA International Advisory Council. She was elected APSIPA's inaugural Distinguished Lecturer 2012-2013 and ISCA Distinguished Lecturer 2015-2016. Her awards include the Ministry of Education Higher Education Outstanding Scientific Research Output Award 2009, Hong Kong Computer Society's inaugural Outstanding ICT Woman Professional Award 2015, Microsoft Research Outstanding Collaborator Award 2016 (1 in 32 worldwide), IEEE ICME 2016 Best Paper Award, IBM Faculty Award 2016, HKPWE Outstanding Women Professionals and Entrepreneurs Award 2017 (1 in 20 since 1999), Hong Kong ICT Silver Award 2018 in Smart Inclusion, and the CogInfoComm2018 Best Paper Award. Helen received all her degrees from MIT. Her research interests include big data decision analytics, and artificial intelligence especially for speech and language technologies to support multilingual and multimodal human-computer interaction. Helen has given invited / keynote presentations including INTERSPEECH 2018 Plenary Talk, World Economic Forum Global Future Council 2018, Taihe Workshop on Building Stakeholder Networks on AI Ethics and Governance 2019 and the World Peace Forum 2019. She has served in numerous Government appointments, including Chairlady of the Research Grants Council's Assessment Panel for Competitive Research Funding Schemes for the Local Self-financing Degree Sector, Chairlady of the Working Party on Manpower Survey of the Information/Innovation Technology Sector (since 2013), as well as Steering Committee Member of Hong Kong's Electronic Health Record (eHR) Sharing. Helen is a Fellow of HKCS, HKIE, IEEE and ISCA.
Dialog is a most fascinating form of human communication. The back-and-forth exchanges convey the speaker's message to the listener, and the listener can derive information about the speaker's thoughts, intent, well-being, emotions and much more. This talk presents an overview of dialog research that concerns our group at The Chinese University of Hong Kong. In the domain of education and learning, we have been recording in-class student group discussions in the flipped-classroom setting of a freshman elite mathematics course. We investigate features in the weekly, within-group dialogs that may relate to class performance and learning efficacy. In the domain of e-commerce, we are developing dialog models based on approximately 20 million conversation turns, to support a virtual shopping assistant in customer inquiries and orders, logistics tracking, etc. In the domain of health and wellbeing, we are capturing and analysing dialogs between health professionals (or their virtual equivalent) and subjects in cognitive screening tests. We also conduct research in both semantic interpretation and dialog state tracking, as well as affective design of virtual conversational assistants. For the former, we have developed a Convex Polytopic Model for extracting a knowledge representation from user inputs in dialog turns by generating a compact convex polytope to enclose all the data points projected to a latent semantic space. The polytope vertices represent extracted semantic concepts. Each user input can then be "interpreted" as a sequence of polytope vertices which represent the user's goals and dialog states. For the latter, we have developed a multimodal, multi-task, deep learning framework to infer the user's emotive state and emotive state change simultaneously. This enables virtual conversational assistants to understand the emotive state in the user's input and to generate an appropriate emotive system response in the dialog turn, which will further influence the user's emotive state in the subsequent dialog turn. Such an affective design will be able to enhance user experience in conversational dialogs with intelligent virtual assistants.
Biography
Helen Meng is Patrick Huen Wing Ming Professor of Systems Engineering and Engineering Management at The Chinese University of Hong Kong (CUHK). She is the Founding Director of the CUHK Ministry of Education (MoE)-Microsoft Key Laboratory for Human-Centric Computing and Interface Technologies (since 2005), Tsinghua-CUHK Joint Research Center for Media Sciences, Technologies and Systems (since 2006), and Co-Director of the Stanley Ho Big Data Decision Analytics Research Center (since 2013). Previously, she served as CUHK Faculty of Engineering's Associate Dean (Research), Chairman of the Department of Systems Engineering and Engineering Management, Editor-in-Chief of the IEEE Transactions on Audio, Speech and Language Processing, Member of the IEEE Signal Processing Society Board of Governors, ISCA Board Member and presently Member of the ISCA International Advisory Council. She was elected APSIPA's inaugural Distinguished Lecturer 2012-2013 and ISCA Distinguished Lecturer 2015-2016. Her awards include the Ministry of Education Higher Education Outstanding Scientific Research Output Award 2009, Hong Kong Computer Society's inaugural Outstanding ICT Woman Professional Award 2015, Microsoft Research Outstanding Collaborator Award 2016 (1 in 32 worldwide), IEEE ICME 2016 Best Paper Award, IBM Faculty Award 2016, HKPWE Outstanding Women Professionals and Entrepreneurs Award 2017 (1 in 20 since 1999), Hong Kong ICT Silver Award 2018 in Smart Inclusion, and the CogInfoComm2018 Best Paper Award. Helen received all her degrees from MIT. Her research interests include big data decision analytics, and artificial intelligence especially for speech and language technologies to support multilingual and multimodal human-computer interaction. Helen has given invited / keynote presentations including INTERSPEECH 2018 Plenary Talk, World Economic Forum Global Future Council 2018, Taihe Workshop on Building Stakeholder Networks on AI Ethics and Governance 2019 and the World Peace Forum 2019. She has served in numerous Government appointments, including Chairlady of the Research Grants Council's Assessment Panel for Competitive Research Funding Schemes for the Local Self-financing Degree Sector, Chairlady of the Working Party on Manpower Survey of the Information/Innovation Technology Sector (since 2013), as well as Steering Committee Member of Hong Kong's Electronic Health Record (eHR) Sharing. Helen is a Fellow of HKCS, HKIE, IEEE and ISCA.
Show speech transcript
