Robust Text-Dependent Speaker Verification via Character-Level Information Preservation for the SdSV Challenge 2020

In our systems, we used two types of front-end networks: TDNN-based [snyder2018xvector] and ResNet-based [chung2019delving] architectures.

In our work, we made use of various objective functions conventionally used in speaker embedding training. Some variants of softmax-based classification loss were employed in our systems. Also, we used end-to-end based losses which directly optimize distance metrics such as Euclidean or Cosine distance. The objective functions used in the systems are as follows:

• •

  Standard Softmax

• •

  Additive Margin Softmax (AM-Softmax) [wang2018amsoft, wang2018amsoft2]

• •

  Additive Angular Margin Softmax (AAM-Softmax) [deng2019aamsoft]

• •

  Angular Prototypical Loss (A-Prototypical) [chung2020in]

• •

  Generalized End-to-End Loss (GE2E) [wan2018ge2e]

In the back-end module, we only used cosine similarity as a scoring method between the two speaker embeddings. No Linear Discriminant Analysis (LDA), Within-Class Covariance Normalization (WCCN), and Probabilistic Linear Discriminant Analysis (PLDA) was applied in this work.

To minimize the domain mismatch (e.g., languages, recording environments, etc.) between the training and the evaluation set and to normalize the distribution of scores during fusion between different models, we used the Adaptive Symmetric Score Normalization (AS-Norm) [matejka2017analysis]. We selected speaker-phrase dependent models in DeepMine Task1 Train Partition (i.e., in-domain training data) as cohort set and used the most similar top 300 scoring files to calculate normalization variables of enrollment and test sets respectively.

Since the TI-SV approaches focus on minimizing the within-speaker variability, the embedding vectors may lack crucial information on the lexical content. Thus such embedding vectors are improper for the TD-SV experiments. To complement the lost contextual discriminability, we introduce a score compensation method. Firstly, we define the posterior of the phrase as follows:

where the set $U=\{u_{j}|\>u_{j}\>is\>j^{th}phrase,\,1\leq j\leq M\}$, $M$ is the number of phrase types in the TD-SV’s dataset, $X$ is an acoustic feature such as MFCC. We estimate the posterior $p(U=u_{j}|X)$ using softmax layers of TDNN-based network. The architecture of this network is identical to the configuration of TDNN-based architecture for CLP but composed of smaller size layers to prevent overfitting (See Table 1). For training the network, we used DeepMine Task1 Train Partition which includes 10 types of phrases. Finally, we compute the compensation factor and the total score between $X$ and $Y$ as follows:

where $s_{X,Y}^{phr}$ is compensation factor, $\tilde{s}_{X,Y}^{spk}$ is the normalized (AS-Norm) score between embeddings of X and Y, $\alpha$ is a scale factor, and $s_{X,Y}$ is the total score between X and Y.

According to the fixed training condition of the challenge, we used the designated datasets for training our systems and utilized RSR2015 dataset [larcher2014rsr] as a validation set for monitoring. The training set for each system was the combination of different datasets and each training dataset is described as follows. DeepMine (Task 1 Train Partition). This is the main dataset, i.e., in-domain data, of the SdSV Challenge. It contains 101,063 utterances from 963 speakers, which have five Persian and five English phrases. We used it for training (1) speaker embedding network and (2) estimating the posterior of phrase, and also as (3) cohort set to calculate parameters of AS-Norm. VoxCeleb1 & 2. We used the development sets of VoxCeleb1 [nagrani2017voxceleb] and VoxCeleb2 [chung2018voxceleb2], which consist of 148,642 and 1,092,009 utterances from 1,211 and 5,994 speakers respectively. In our systems, they were used to train the speaker embedding networks. LibriSpeech. To train the CTC-based ASR model, namely $Jasper$, which was utilized in character-level pooling and for estimating the posterior of phrase, we used the train-clean/other sets of LibriSpeech corpus [panayotov2015librispeech], which comprise 281,241 utterances from 2,338 speakers. Additionally, in some systems, we employed them for training speaker embedding networks.

According to the trial condition of the challenge, the enrollment was accomplished using three utterances of a specific phrase for each model and among four types of trials in the TD-SV task, only Target-Correct, where the target speaker utters the correct pass-phrase, was considered as target and the rest was an imposter. The trial set was divided into two subsets: a progress subset (30%), and an evaluation subset (70%). The progress subset was used to monitor progress on the leaderboard, while the evaluation subset was used for the official results.

We analyzed two experimental scenarios. First, we verified the feasibility and effectiveness of the character-level pooling strategy for the TD-SV task through the results on the progress and evaluation subsets (Table 3). In the second experiment, we applied AS-Norm and score compensation to all the systems we used in the first experiment, to further boost the performance. Also, we fused different systems and confirmed the best primary system and single system on the progress and evaluation subsets in terms of MinDCF, which was the main metric for the challenge (Table 4). No preprocessing such as data augmentation or VAD was applied to the training and trial data. Analysis of character-level pooling strategy (Table 3). Each subsystem (1-15) was composed of different front-ends, pooling techniques, objectives, and training datasets described in Section 2 and 3.1. Among them, system (5) utilized text-dependent training (TDT), which was jointly trained by combined classes of speaker and phrase (i.e., speakers $\times$ phrases classes). In systems of (5-16), phrase information was not considered, since they were trained for TI-SV. For the reasons stated in Section 2.2, the character-level pooling methods (systems 1-4) showed improved performances compared with other systems in terms of 0.3554 MinDCF and 8.48% EER on the challenge’s evaluation subsets (See Table 3). Results using AS-Norm & score compensation (Table 4). We used AS-Norm and score compensation described in Sections 2.6 and 2.7 respectively, for improvement of performances. As you can see in Table 4, the performance of all systems was improved significantly. In particular, the performances of systems that didn’t consider the lexical context (5-15) increased greatly, compared to the character-level pooling systems (1-4), which showed minor improvement. The best performance of a single system was 0.1307 MinDCF and 3.18% EER on the evaluation subsets. From these results, we could interpret that the better the speaker is distinguished, the higher the performance can be achieved, when score compensation was applied. Finally, we performed the fusion by computing the equal-weighted sum of the scores of different systems, which were TDNN-based (18), ResNet-based (19), and all systems (20). Overall, the performance of ResNet-based systems outperformed TDNN-based systems in the case of both single systems and fusions, and the best primary system was the fusion of all systems. It obtained 0.0785 MinDCF and 2.23% EER on the evaluation subset.