AIRCADE: an Anechoic and IR Convolution-based Auralization Data-compilation Ensemble

Reverberation is the persistence of sound in a space after the source ceases to emit it. It is mainly caused by reflections of the sound waves off the surfaces within the space and gradually dissipates over time. Since the characteristics of the space in which reverberation occurs can influence the duration and intensity of this phenomenon, the Reverberation Time (RT) is considered an important aspect of room acoustics (Beranek (2004); Lapointe (2010)).

In the context of entertainment, reverberation can affect the Human Auditory System (HAS) in positive ways, since some degree of it can help to enhance the perceived loudness and richness of sounds. This occurs because the multiple reflections can create a sensation of spaciousness and immersion, which can be aesthetically pleasing, particularly in music. On the other hand, in the context of communications, excessive reverberation can affect the quality and intelligibility of speech because the multiple reflections can create a "smearing" effect that can make it difficult to distinguish individual sounds and syllables. Moreover, the reduction of the overall Signal-to-Noise Ratio (SNR) of a given sound source can also mask quiet sounds, making it harder to understand them (Gelfand (2017); Lyon (2017)).

Therefore, the effects of reverberation on the HAS can vary widely, depending on the duration and intensity of the phenomenon, the frequency content of the sound source, the individual characteristics of the listener’s hearing, etc. In general, it is desirable to optimize the amount of reverberation in a given acoustic scenario to ensure the best possible listening experience for the intended audience. In this situation, dereverberation is a process that can be used to reduce or remove the effects of excessive reverberation from an audio signal. This is typically used in situations where the audio signal has already been recorded in a reverberant environment, and the resulting reverberation is unwanted or detrimental to the quality of the recorded audio (Naylor et al. (2010)).

In recent years, data-driven dereverberation methods have become increasingly popular due to their ability to learn complex mappings between the anechoic and reverberant signals. While showing promising results, these methods also have a wide range of applicability, including speech recognition, speaker verification, music production, etc (Xu et al. (2014); Hershey et al. (2016)).

Still, an important challenge for these methods is the need for large amounts of training data, with paired clean and reverberant audio signals for training, which, ideally, should cover a wide range of acoustic environments, microphone types, and speaker characteristics, to ensure that a model will generalize well to new scenarios. Examples of dereverberation datasets include the REVERB challenge dataset (Kinoshita et al. (2013)), BUT Speech@FIT Reverb Database (Szöke et al. (2019)), and VoiceBank-SLR (Fu et al. (2022)), among others.

However, most of these datasets carry some limitations, such as narrow-band anechoic data, i.e., only speech is considered as a signal of interest, and absence of Impulse Response (IR) data, i.e., usually only anechoic and reverberant data are paired for supervised training. Hence, in this paper, we introduce a new data-compilation ensemble, primarily intended for training data-driven dereverberation models capable of dealing with full bandwidth audio signals, e.g., speech, song, music etc. We offer pairs of natural anechoic and IR data, compiled from datasets licensed under Creative Commons Attribution 4.0 International License, together with Python scripts for convolution-based auralization, under the hypothesis that these ensembles could serve as better training and evaluation tools for such algorithms.

The remainder of this paper is organized as follows: Section 2 details the methodology used for selecting the anechoic and IR data, and then synthesizing the auralized data. Section 3 describes the resulting auralized data and their annotations. Finally, Section 4 presents an overall discussion of our obtained results, together with some pertinent considerations to conclude our study.

To simulate or recreate an acoustic environment, such as a concert hall or a recording studio, using computer algorithms and specialized software, one can resort to a well known technique called auralization. It involves measurements or simulations of the space’s physical properties to mimic its acoustic characteristics. To proceed accordingly, there are various techniques available, such as acoustic ray tracing and convolution-based auralization (Kleiner et al. (1993)).

From a signal processing point of view, the convolution is a mathematical operation that describes the interaction between two signals. In the context of auralization, convolution can be used to simulate the effects of an acoustic environment on an audio signal. The basic idea is to convolve an audio signal with an IR that describes the acoustic characteristics of a room or space. The resultant output produces a new signal that represents the original audio signal after it has been modified by the acoustic characteristics of the room (Allen and Berkley (1979); Oppenheim et al. (1997)).

In the context of data-driven dereverberation methods, the choice for either natural, synthetic or joint datasets, i.e., those obtained by processing combinations of natural recordings with synthetic sounds, directly impacts in-the-wild applicability. This occurs because, when models are trained on mostly synthetic data, they usually don’t generalize well for real-world scenarios. However, the majority of recent studies use joint combinations, e.g., by convolving anechoic data with naturally recorded or synthesized IR data etc, perhaps because it would be too cumbersome to naturally acquire all the necessary data (dos Santos et al. (2022)). Hence our dataset attempts to reach a balance between these features, compiling real anechoic and IR data, and then synthetically producing auralized data ensembles by means of convolution.

The processed anechoic and IR data is hosted at Zenodo, with an approximate total file size of 1.3 GB. For simplicity, all samples in our data-compilation were renamed, e.g., $guitar\_0000$, $rir\_0000$, $song\_0000$, $speech\_0000$, and so on. To synthesize different versions of the auralized data ensemble, the reader is referred to GitHub, where Python scripts are available for downloading the base data-compilation and synthesizing a chosen version of the auralized data ensemble. Table 1 illustrates the differences between all versions, detailing the number of song, speech, guitar, IR and auralized samples in each one, together with their respective total file size and duration.

Overall, the data-compilation ensemble presented in this work provides a diverse and comprehensive set of acoustic scenes for use in dereverberation tasks, as well as some other audio signal processing applications, such as emotion recognition and MIR. By combining different types of signals of interest, including speech, song, and acoustic guitar sounds, with a variety of IRs, we provide a challenging dataset for researchers working on dereverberation and related fields.

The dataset is available in different sizes, from a tiny version with limited data, to a large version with almost $300,000$ samples, allowing users to choose the most suitable version for their specific research needs. The dataset also includes metadata that can be used to trace back each sample to its original annotations, facilitating the use of the dataset for tasks such as emotion recognition and MIR.

We hope that this dataset will be useful for researchers working on dereverberation and related fields, and we encourage its use in future research. We also believe that the diversity and variability of the dataset can facilitate the development of more robust and generalizable algorithms for dereverberation and other audio signal processing tasks.