Towards Robust Transcription: Exploring Noise Injection Strategies for Training Data Augmentation

Recording piano performances in uncontrolled environments often results in widely varying SNRs due to lack of high quality equipment, background noise, reverberation, and other environmental factors. In such settings, the SNR tends to be lower than in controlled professional environments such as studios. To rigorously assess the robustness of the models under these conditions, we evaluated their performance across a broad range of SNR values, ranging from -6 dB to 45 dB, in 3 dB intervals, covering a total of 18 different SNR levels.

For this experiment, we utilized two state-of-the-art models: Onsets and Frames [5] and the model proposed by Kong et al. [8]. The Kong et al. model was tested using a publicly available pre-trained checkpoint,²²2https://github.com/bytedance/piano_transcription while the Onsets and Frames [5] model was re-implemented using the PyTorch framework,³³3https://github.com/jongwook/onsets-and-frames following the original training procedures and specifications outlined in the respective papers.

We generated white noise-augmented versions of the MAESTRO v3 [7] dataset for inference. The test split, consisting of 177 recordings, was used for evaluation as defined in the dataset’s metadata. To create the augmented data, we calculated the power of each target audio sample and added scaled white noise to achieve the desired SNR levels. The resulting audio was then normalized to retain the RMS value of the original, noise-free recording. In instances where clipping occurred, we adjusted the affected sections to the maximum representable value to prevent distortion, ensuring that the integrity of the audio signal was preserved throughout the evaluation process.

Figure 1 presents the overall results of this experiment. As the SNR level decreases, a clear decline in model performance is observed. Both the Kong et al. [8] model and Onsets and Frames [5] show a reduction in F1 scores compared to their baseline performance on clean data (96.7% and 94.5%, respectively), with around a 5% relative drop at 12 dB SNR and around a 10% relative drop at 9 dB SNR.

To investigate whether performance degradation in noisy environments can be mitigated through data augmentation, we employed the Onsets and Frames model [5], training it with white noise-injected audio. We introduce the term Clean-to-Noise Ratio (CNR) to represent the proportion of clean to noise-injected audio sampled during training. Based on the hypothesis that the level of perturbation impacts performance, we trained the model using CNR levels of 0 (fully perturbed), 1/3, 1, 3, and $\infty$ (clean audio only). For each CNR, the model probabilistically samples from the clean or noisy audio. In case of noisy audio, the SNR dB value is randomly sampled from [0, 24], followed by RMS normalization and clipping prevention. This SNR range was chosen based on prior research [7, 13] and results from Sect. 2.1, where significant performance degradation was observed.

We followed the original Onsets and Frames [5] configuration throughout the training process, standardizing the number of iterations to 100k for all experiments. This decision was based on preliminary experiments with noise-injected audio, which showed that increasing the number of iterations beyond 100k did not result in lower losses.

The performance evaluation of the trained models is presented in Figure 2.

We observe that the the systems trained with noisy data all considerably outperform the system trained with clean data for low SNRs. As the SNR increases, the gap decreases, and for very high SNRs, we observe a tendency for the highly perturbed training scenarios (CNR $\nicefrac{{1}}{{3}}$, 0) to be outperformed by the system trained with clean data (CNR $\infty$), which in turn is slightly outperformed by the slightly perturbed training scenarios (CNR 3, 1).

To assess whether these observations are statistically significant, we conducted a series of t-tests. This analysis was performed across four distinct CNR values. Table 1 summarizes the ranges where statistically significant differences (t-statistic < 0 and p-value < 0.05) were observed in Precision, Recall, and F1 scores.

These findings indicate that while training with noise-injected data can enhance model robustness, excluding clean data entirely (CNR=0) leads to performance degradation in acoustically controlled environments, such as when testing on clean data (SNR=$\infty$). Thus, the absence of clean data during training adversely impacts performance under clean conditions. Moreover, the percentage of perturbed training samples is an important parameter of augmentation with impact the improvement gained and the SNR range of the improvement.