End-to-End Language Diarization for Bilingual Code-switching Speech
(3 minutes introduction)
| Hexin Liu (NTU, Singapore), Leibny Paola García Perera (Johns Hopkins University, USA), Xinyi Zhang (NTU, Singapore), Justin Dauwels (Technische Universiteit Delft, The Netherlands), Andy W.H. Khong (NTU, Singapore), Sanjeev Khudanpur (Johns Hopkins University, USA), Suzy J. Styles (NTU, Singapore) |
|---|
We propose two end-to-end neural configurations for language diarization on bilingual code-switching speech. The first, a BLSTM-E2E architecture, includes a set of stacked bidirectional LSTMs to compute embeddings and incorporates the deep clustering loss to enforce grouping of languages belonging to the same class. The second, an XSA-E2E architecture, is based on an x-vector model followed by a self-attention encoder. The former encodes frame-level features into segment-level embeddings while the latter considers all those embeddings to generate a sequence of segment-level language labels. We evaluated the proposed methods on the dataset obtained from the shared task B in WSTCSMC 2020 and our handcrafted simulated data from the SEAME dataset. Experimental results show that our proposed XSA-E2E architecture achieved a relative improvement of 12.1% in equal error rate and a 7.4% relative improvement on accuracy compared with the baseline algorithm in the WSTCSMC 2020 dataset. Our proposed XSA-E2E architecture achieved an accuracy of 89.84% with a baseline of 85.60% on the simulated data derived from the SEAME dataset.
