Noisy Neonatal Chest Sound Separation for High-Quality Heart and Lung Sounds

Accurate and timely assessment for signs of serious health problems such as cardio-respiratory diseases is an essential requirement to provide care to newborns [19]. Recording chest sounds with a stethoscope is a common and simple method to obtain such information. In recent times, the availability of digital stethoscopes for neonates has attracted several studies [20, 21, 22]. Existing digital stethoscopes can connect with smartphones for mobile health and can enable remote healthcare through telehealth. However, a higher level of noise in neonatal intensive care in comparison to adult and paediatric wards has resulted in poor quality chest sound recordings and inaccurate assessment. For instance, estimation of heart rate and breathing rate from low-quality signals is error-prone [3, 6].

Noise interference in chest sounds can be broken up into four groups. Firstly, there are external noises, such as crying, stethoscope movement, talking and general background noise in neonatal intensive care [23]. Secondly, while heart and lung sounds are both diagnostically important, they act as noise sources for one another. Thirdly, other internal body sounds such as bowel sounds, gastric reflux, and air swallowing. Finally, neonates, particularly ones born earlier than 32 weeks, can experience respiratory-related conditions, requiring respiratory support such as high flow, ventilator or bubble Continuous Positive Airway Pressure (CPAP), which are major sources of interference. Overall, it is essential to reduce these noises, and separate heart and lung sounds before any assessment and diagnosis.

Denoising and sound separation methods to obtain high-quality heart and lung sounds can be broken up into multi-channel and single-channel methods. In multi-channel methods, a reference signal such as an electrocardiogram or secondary microphone placed either to capture external noise and/or secondary chest recording is used [24]. Overall, these methods require additional sensors, which are not always accessible and feasible to implement.

For single-source sound separation and denoising, current methods have proven only partially effective. Table I summarises existing single-source sound separation methods and Section IV compares them. Many of the methods utilise heart sound segmentation to obtain S1 and S2 peaks. As shown in our past works, heart sound segmentation accuracy drops significantly for low-quality recordings [3, 6]. Overall this means many of these methods are only effective in low-noise scenarios. For lung sound separation, this has proven difficult due to its large frequency band that overlaps with heart and noise sources.

Another limitation with past works is that they rely on adult-based parameters, which are not suitable for the separation of neonatal chest sounds. To address this, existing methods that rely on heart sound segmentation, utilise a modified version suitable for neonates, which was developed in our past work [3]. Additionally, instead of a single-value parameter being utilised, a set of relevant parameters suitable for neonatal chest sound separation are considered and tested, as highlighted in Table I. Finally, for singular spectrum analysis (l) specifically, an additional constraint of top eigenvalue pairs having the strongest frequency component less than 250 Hz are assigned as heart components is added. This constraint was added to address the issue of misclassification of lung components as heart, as they also contain semi-periodic components.

This paper presents a new approach, based on Non-negative Matrix Factorisation (NMF) and Non-negative Matrix Co-Factorisation (NMCF) to obtain high-quality heart and lung sounds. The model is trained with high-quality heart, lung and noise sound examples. In NMF this training occurs beforehand, whereas in NMCF this training occurs in parallel with separating sounds from the noisy recording into heart, lung and noise. The training set enables the utilisation of more detailed information about the frequency and temporal aspects of heart, lung and noise. Whereas for NMCF, the parallel training and separation of sounds from the noisy recordings enable the model to adapt more specifically to that recording. Overall, this method enables higher quality lung and heart sounds to be generated for analysis.

A preliminary version of the NMCF work has been reported, which utilised high-quality heart and lung sounds, with no reference noise sound examples [9]. Initial results on a real-world dataset showed it was superior to existing NMF methods [9].

Four key contributions are presented in this paper. First, existing single-source denoising and heart and lung sound separation methods originally developed for adults were adapted and implemented on newborn chest sound recordings. Second, new NMF and NMCF approaches focused on obtaining high-quality heart and lung sounds specifically for the newborn population are proposed. Third, we incorporate a noise component in the NMF and NMCF models, to separate the sounds into not only heart and lung sounds but also noise sounds. Finally, the methods are assessed using artificial and real-world noisy neonatal chest sounds with heart and lung signal quality, and heart and breathing rate accuracy.

The rest of this paper is organised as follows. Section II provides the background of proposed NMF and NMCF methods. Section III presents a detailed implementation of the methods and their evaluation. Results and discussion are provided in Sections IV and V, respectively. Section VI concludes the whole work.

NMF decomposes a given non-negative matrix $V\in\Re^{F\times T}_{+}$ into two non-negative matrices $W\in\Re^{F\times K}_{+}$ and $H\in\Re^{K\times T}_{+}$ (1), where $K<min(F,T))$ and E represents the reconstruction error between V and WH.

In denoising and sound separation, V represents the magnitude of the time-frequency representation of the recording mixture [9].

The basis matrix, W contains the basis column vectors $w_{1}$ to $w_{K}$ that represent the spectral pattern of different types of signals sources (e.g. heart, lung and noise) or their sub-components. The activation matrix H, contains the temporal activation row vectors $h_{1}$ to $h_{k}$, that represents when the signal sources occur during a particular time frame. These sub-components can be combined such that the first set of components (1 to $b_{h}$), second set of components ($b_{h}+1$ to $b_{h}+b_{l}$) and third set of components ($b_{h}+b_{l}+1$ to $b_{h}+b_{l}+b_{n}$) represent heart ($V_{h}=W_{h}H_{h}$), lung ($V_{l}=W_{l}H_{l}$) and noise respectively ($V_{n}=W_{n}H_{n}$) [9].

For supervised and semi-supervised NMF, the basis matrix W is optimised with reference heart, lung and noise sounds during the training phase. The basis matrix W is then fixed and H is optimised with the noisy mixture recording during the test phase. In NMCF, instead of having a training and test phase, the basis matrix W is optimised simultaneously with sound separation. This method enables more efficient sound separation as the mixture recording can also contribute to the training of W [9].

However, as obtaining pure heart or lung sounds is not feasible, we propose a modified version of NMF and NMCF (equations (2) and (3), respectively) as described in Algorithm 1, and Figure 1. In this version, datasets of high-quality heart, lung, and noise sounds are used in the cost function to enable the generalisation of $W_{h}$, $W_{l}$ and $W_{n}$ respectively for the sound separation. Note, these reference datasets are generalisable and are not obtained from the same subject as the noisy mixture recording that is being denoised.

In both the proposed NMF and NMCF equations (2) and (3), the matrices W and H are optimised by minimising the cost function D (4). $D_{\beta}$ refers to $\beta$-divergence cost function, with the most popular values being $\beta=0,1,2$ [9]. A sparsity penalty on the activation matrix H is calculated based on the L1-norm of H, and $\mu$ controls the importance of the sparsity constraint. The sparsity penalty enables more detailed decomposition both temporally and spectrally while ensuring only a small set of meaningful basis vectors are active at a single time frame [25].

For proposed NMF (2) we have introduced a cost function that utilises datasets of clean heart, lung and noise sounds that are obtained from different subjects than the noisy mixture recording. This differs from past work which either requires simultaneous reference recordings from the same subject or relies on blind decomposition [24, 9].

While for proposed NMCF (3), building on our past work, we have introduced a supervised noise component [9]. The weighting factors $\lambda_{h}$, $\lambda_{l}$ and $\lambda_{n}$ represent the level of co-factorisation and are treated as hyperparameters. Additionally, an unsupervised component $W_{un}$ is added to deal with the large variety of noises that are not covered, therefore avoiding these components being assigned to the heart or lung components.

Based on the cost function in (4), the multiplicative update rule for W and H are shown in (5) and (6) respectively. Note that division and $\otimes$ refer to element-wise division and multiplication.

Figure 3 and Table III show the artificial and real-world chest sound separation results respectively. Methods (a), (a un), (b), (b un) are proposed NMCF, NMCF with Unsupervised Components, NMF and NMF with Unsupervised components methods respectively. Methods (c) to (n) are existing methods, as specified in Table I.

Figures 3(a) and 3(b) show the artificial mixture sound separation results for heart and lung sounds, as detailed in Section III-D1. Overall for heart and lung sound separation, both the proposed NMCF (a) and NMF (b) methods significantly outperform existing methods in all situations, with median SDR improvement ranging from 2.7 dB for respiratory support noise to 11.6 dB for general noise compared to next best existing method.

For heart sound separation, in the no noise case, the proposed NMCF (a) and NMF (b) methods outperforms all existing methods except adaptive line enhancement (d), which has a median SDR improvement of 1.5 dB and 0.8 dB over proposed methods respectively. However, adaptive line enhancement (d) produces minor temporal distortions in the separated heart sound, resulting in significantly lower SI-SDR values of 18.1 dB and 17.9 dB for proposed NMCF (a) and NMF (b) respectively. For the general noise case, proposed NMCF (a) and NMF (b) methods outperform the next best existing method by 4.3 dB and 4.0 dB for cry noise and 1.7 dB and 2.1 dB for stethoscope movement noise, for median SDR improvement. Similarly for respiratory support noise case, proposed NMCF (a) and NMF (b) methods outperform the next best existing method by 1.8 dB and 1.4 dB for bubble CPAP noise and 1.8 dB and 1.0 dB for ventilator CPAP noise.

For lung sound separation, in the noise case, the proposed NMCF (a) and NMF (b) outperformed the next best existing method by 0.9 dB and 0.8 dB (not significant p-value=0.055), this increases to 3.1 dB and 1.4 dB when for hard to separate situations (lung SNR less than -10 dB). For the general noise case, proposed NMCF (a) and NMF (b) methods outperform the next best existing method by 8.4 dB and 6.9 dB for cry noise and 0.3 dB and 0.7 dB for stethoscope movement noise. For high stethoscope movement noise (SNR greater than 0 dB) median SDR improvement increases to 0.5 dB and 0.9 dB. For respiratory support noise case, proposed NMCF (a) and NMF (b) methods outperform next best existing method by 0.6 dB and 0.0 dB (not significant p-value=0.29) for bubble CPAP noise and 1.0 dB and 0.4 dB for ventilator CPAP noise. For high respiratory support noise (SNR greater than 0 dB), median SDR improvement increases to 1 dB and 0.1 dB (not significant p-value=0.25) for bubble CPAP noise and 1.2 dB and 0.9 dB for ventilator CPAP noise.

As both proposed NMCF (a) and NMF (b) methods do not produce temporal distortions, SI-SDR results are comparable with SDR. Both methods significantly outperform existing methods in all situations, with median SI-SDR improvement ranging from 5.2 dB for respiratory support noise to 9.9 dB for general noise compared to the next best existing method. For specifically heart sound separation or lung sound separation, again, both proposed methods significantly outperformed all existing methods in all situations for SI-SDR improvement.

For SIR results, both proposed methods significantly outperform existing methods in all situations, with median SIR improvement ranging from 3.8 dB for respiratory support noise to 8.2 dB for general noise compared to the next best existing method. For heart sound separation, empirical mode decomposition (e) significantly outperformed both proposed methods in the no noise situation and was comparable for the general noise case. For lung sound separation, the filtering (f) method was comparable to the proposed NMF method for no noise case, but was significantly inferior for hard to separate situations (lung SNR less than -10 dB). For all other scenarios, both proposed methods significantly outperformed existing methods for heart and lung sound separation.

For existing methods, the best parameters based on artificial dataset SDR values are as follows:

• •

  Adaptive Fourier Decomposition (c): Decomposition level of 5 (Cry Noise) and 10 (No Noise, Stethoscope Movement Noise, Respiratory Support Noise)

• •

  Adaptive Line Enhancement (d): Delay of 1 (No Noise, Respiratory Support Noise), 10 (Stethoscope Movement Noise) and 50 (Cry Noise)

• •

  Empirical Mode Decomposition (e): Decomposition using ensemble empirical mode decomposition (All Cases)

• •

  Interpolation (g): Remove entire segments containing heart sounds and interpolate (All Cases)

• •

  Modulation Filtering (h): bandstop of 3-20 Hz (All Cases) for heart sounds and bandpass of 4-20 Hz (No Noise, Ventilator CPAP and General Noise) and 6-20 Hz (Bubble CPAP Noise) for lung sounds

• •

  Wavelet Transform Based Stationary Non-Stationary Filter (m): Adaptive threshold of 3 (No Noise, Respiratory Support Noise, Cry Noise) and 3.5 (Stethoscope Movement Noise)

For proposed NMCF (a) and NMF (b) methods, the best parameters based on artificial dataset SDR values are as follows:

• •

  Time-Frequency Representation = STFT, FFT Size = 1024 samples, Window Size = 512 samples and Hop Size = 256 samples (NMCF All Cases, NMF No Noise, General Noise, Bubble CPAP Noise). Time-Frequency Representation = STFT, FFT Size = 2048 samples, Window Size = 2048 samples and Hop Size = 512 samples (NMF Ventilator CPAP Noise)

• •

  Beta loss = 1 (All Cases)

• •

  Sparsity penalty = 0.1 (NMCF All Cases, NMF Stethoscope Movement Noise and Respiratory Support Noise), 0 (NMF No Noise and Cry Noise)

• •

  NMCF $\lambda_{h}$ = 0.25 (No Noise), 0 (General Noise, Respiratory Support Noise)

• •

  NMCF $\lambda_{l}$ = 0 (All Cases)

• •

  NMCF $\lambda_{n}$ = 0.25 (Stethoscope Movement Noise, Ventilator CPAP Noise), 0.75 (Cry Noise, Bubble CPAP Noise)

Table III shows the real-world chest sound separation results, as detailed in Sections III-D2 and III-D3. Overall for heart and lung sound separation, all proposed methods significantly outperform existing methods in all situations. Mean signal quality improvement ranged from 0.40-0.63 for no respiratory support case and 0.74-1.12 for respiratory support case compared to the next best existing method.

For heart sound separation, all proposed methods outperform existing methods. Mean signal quality improvement compared to the next best existing method ranged between 0.10-0.17 for no respiratory support case and 0.22-0.38 for the respiratory support case. All signal quality improvement values were significant except the proposed NMCF (a) with the no respiratory support case (p-value=0.06).

For lung sound separation, all proposed methods significantly outperformed existing methods except filtering (f). Mean signal quality improvement compared to the filtering method (f) ranged from -0.04-0.14 for the no respiratory support case and 0.00-0.27 for the respiratory support case. For no respiratory case, only the proposed NMF method (b) was significantly better than the filtering method (f). Whereas for the respiratory case, both the proposed NMCF (a) and NMCF with unsupervised components (a un) significantly outperformed the filtering method (f).

As seen in Table III vital sign improvement results were very sporadic. In general, all methods had a median vital sign error improvement of 0, but were positively skewed, as seen in the mean values. Due to the large variation in results, many of the comparisons between methods were not significant. For no respiratory support, both NMCF methods, (a) and (a un), performed best for heart rate estimation (not significant for NMF clustering 2 (j) and filtering (f)). Whereas NMF without supervised components (b un) performed best for breathing rate estimation (not significant for filtering (f) and modulation filtering (h)). For the presence of respiratory support noise, the existing filtering method (f) performed significantly better than all proposed methods for heart rate estimation and wavelet transform based stationary non-stationary filter (m) performed significantly better than proposed NMCF methods, (a) and (a un).

Overall, both proposed NMCF and NMF methods performed well, producing comparable or superior results to existing methods. In the artificial dataset, NMCF had a median SDR improvement of 0.00, 1.71, -0.55, 1.17 and 0.92 dB for no noise, cry noise, stethoscope movement noise, bubble CPAP noise and ventilator CPAP noise, respectively in comparison to NMF. Whereas in the real-world dataset, NMF outperformed NMCF in the no respiratory support case, and NMCF outperformed NMF in the respiratory support case. As seen in the results, in the presence of hard-to-separate noise such as respiratory support noise, co-factorisation in the NMCF method aids in the better adaption for effective sound separation, as opposed to the NMF method. Whereas, in the general case of just heart-lung sound separation, the NMF method is sufficient for successful sound separation. As the proposed NMF method can be implemented in real-time, it would be recommended to utilise this method initially. If the signal quality is still poor after the implementation of NMF due to hard-to-separate noise, then the proposed NMCF method should be utilised afterwards. NMCF allows for better adaptation of the basis vectors, as opposed to fixed basis vectors in the NMF method, enabling better ability to segregate noise.

Vital sign error results were less conclusive and promising for the purposed methods. With regards to median vital sign error improvement being 0, in general, the vital sign estimation methods are designed to be robust to an extent to noise. In particular, the heart rate estimation method was designed specifically to deal with noisy heart sounds [2]. Instead, the sound separation and denoising methods mainly only improved results when there was a major vital sign estimation error in the raw recordings, explaining the large standard deviation and positive mean seen in the results. For sound separation in the presence of respiratory support noise, the proposed NMF and NMCF methods were inferior to existing methods. One possible explanation for this is because after separation, with respiratory support noise partially removed, there were cases where heart and lung sounds were not present, resulting in an underestimation in vital signs. Whereas, for other methods that do not explicitly remove the respiratory support sounds, these sounds were miscounted as heartbeats/breathing periods, avoiding the underestimation in vital signs. These conclusions are further supported by existing methods that poorly estimated vital signs in no respiratory support case showing improvement in the more difficult respiratory support case. In particular for lung sounds, all methods struggled to successfully obtain clean lung sounds for the respiratory support case, suggesting all breathing rate error improvements could be inaccurate. Future work is required to accurately determine if these removed respiratory support sounds are also unintentionally removing these heartbeats/breathing periods, or if there is a subset of recordings in which heart and lung sounds cannot be recovered.

As seen in Table III, the introduction of the unsupervised noise component led to an improvement in signal quality results and a decrease in signal quality for the NMF method. For the NMCF, the benefit of the unsupervised noise component can be explained by the fact that it prevents co-factorisation and learning of the basis vectors and temporal activation to be heavily affected by noise that was not present in the reference sound set. Instead, other noise sources are allocated to the unsupervised component as they are not related to any of the reference sounds. For NMF, the basis matrix has already been pre-computed and fixed for the sound separation phase. Hence, this method is not as affected by the presence of other sources of noise, especially if they have fairly distinct frequency components. Instead, relevant heart and lung sound components can be misplaced into the unsupervised components, due to the unsupervised nature of calculating these basis vectors, resulting in a decrease in performance. This is a key limitation of the introduction of unsupervised components, as whilst it can remove unknown noise sources, it can also remove some of the heart and lung sound components.

Whilst our proposed sound separation methods worked better or comparably to existing methods for respiratory support noise, there is still a large room for improvement. In both the artificial dataset and real-world recordings, respiratory support noise was still present in the separated sounds. This can be explained by the fact that respiratory support noise has a large frequency overlap with the desired heart and lung sounds. This large frequency overlap results in the frequency basis vectors being quite similar for the respiratory support noise and heart and lung sounds. Some possible ways to improve sound separation in the presence of respiratory support noise is to have additional reference sounds and add a temporal constrain or decomposition. Currently, only 5 and 8 reference sounds were provided for bubble and ventilator CPAP respectively, which may not provide the entire variety of different flow rates and sounds that can occur. With regards to temporal constraint or decomposition, respiratory support noise tends to be present for the entire recordings, whereas heart and lung sounds occur on and off. Therefore, there is a potential for reassignment of sub-components after NMF or NMCF based on temporal activation, similar to past works [18, 32]. Additionally, instead of reassignment post decomposition, there is the possibility of adding a temporal constraint or cost function, which would enable the decomposition of the components based on temporal activation. However, future work would be required to develop this temporal decomposition method.

With regards to computational cost, the proposed NMCF method is not suitable for real-time processing, whereas the proposed NMF method is. Given, for the most part, the proposed NMF method produces superior results to all existing methods and comparable results to the proposed NMCF method, this may be sufficient. There are a couple of ways to improve the computation speed of the proposed NMCF method. Firstly, as the optimal heart and lung sound co-factorisation weights were zero (i.e., no co-factorisation with mixture chest sound recording), a fusion of the NMF and NMCF could be employed. For these zero co-factorisation weights, instead of calculating the associated basis vectors during the separation of the mixture chest sound, instead, similar to NMF, the basis matrices of heart and lung sounds can be pre-computed and fixed. Accordingly, the computation time would be reduced by up to 36% potentially. Secondly, the pre-computed base matrices could be used as the initialisation of the basis matrices for the proposed NMCF method. This initialisation would place the basis matrices closer to the optimal result, meaning fewer iteration of the update algorithm would be required. Thirdly, there is still room for optimisation of the number of reference examples, the number of bases for each sound component, FFT size and hop size, all of which could reduce the size of the basis and activation matrices, overall reducing the computation time. Finally, the multiplicative update algorithm could be optimised, for instance using C code MEX functions, as opposed to MATLAB.

By and large, all these conclusions on computational time with regards to real-time processing is specific to the laptop parameters used in this particular study. It would be expected similar computational times for high-end laptops, general desktops, or a general device connected to cloud computing, however, computational times would be expected to increase if these sound separation methods were used on a smartphone with its onboard processing power.

An artificial dataset was proposed to quantitatively assess the effectiveness of sound separation with heart, lung and various noise sounds. However, there are several limitations with this dataset. Firstly, to obtain clean reference sounds, bandpass filtering of 50-250 Hz and 200-1000 Hz was applied to heart and lung sounds. Similarly, for cry noise, highpass filtering of 300 Hz was applied. Whilst these frequency bands contain the majority of the heart, lung and cry sounds, there can still be prominent sounds outside these frequency bands. Therefore, to more accurately replicate the larger frequency overlap between these sounds and assess the sound separation methods, pure and unfiltered reference sounds would be required. However, this is quite difficult to obtain with newborns. For example, clean reference heart sounds are typically acquired from adults by asking them to hold their breath, which is not feasible with neonates. However neonates, in particular preterm neonates, have sporadic breathing patterns of fast periodic breathing followed by a period of no breathing. Therefore, it may be possible to obtain short segments of clean heart sounds. Obtaining clean reference lung sounds is more difficult as heart sound noise will always be present. Future collection of chest sounds with digital stethoscope placement on the chest but further away from the heart or on the back may minimise heart sounds and enable clearer lung sounds. Another option to obtain clean reference sounds with a full frequency range is to artificially generate the sounds with the desired properties. Past works on modelling adult heart sounds have been done and could be transferable for neonates. Similarly, a model of neonatal lung sounds and various noise sources could be created.

The second limitation with the artificial dataset is how cry noise was mixed with lung sounds. As inspiration and expiration cannot occur simultaneously as a newborn is crying, this should be taken into account. While for stethoscope disconnect noise, this was resolved by zeroing out the section of the heart-lung sound mixture that it was going to be inserted, this is not appropriate for cry noise. For a more appropriate lung sound and cry noise mixture, the periods and order of inspiration and expiration should be taken into account and inserted reasonably. However, as lung sounds and breathing rate change while a baby is crying, it may not even be useful or necessary to try and accurately represent this mixture.

The third limitation with the artificial dataset is the relatively small evaluation dataset, in particular for respiratory support noise and heart and lung sounds. For respiratory support noise, future collections of pure respiration support noise obtained by placing the stethoscope on the breathing tube will be acquired. Whereas the limitation with heart and lung sounds is not the number of chest sound recordings available, but the difficulty in obtaining clean sounds. Again, one potential solution to this is to artificially create heart and lung sounds.

Furthermore, signal quality estimation of real chest sounds was proposed to quantitatively assess the effectiveness of sound separation on real-world data. This is an essential assessment as it overcomes many of the limitations with artificial datasets and assesses the sound separation methods on the actual data it would be used for. However, there are also several limitations with the proposed signal quality estimation method. Firstly, while the method attempts to objectively assess heart and lung sound quality, it is not 100% accurate [3, 6]. Therefore, individual comparison of methods with a single recording may not be appropriate due to misclassification errors with the signal quality estimation model. Whereas, overall trends on signal quality improvement and comparison of methods is more appropriate. Secondly, the method assesses the overall quality of the heart and lung sounds. This overall assessment does not directly take into account if significant portions of the heart or lung sound components are also being removed to obtain the ”noise-free” sounds. These limitations can also partially explain why the artificial dataset vs real-world dataset sound separation methods differs.

This paper has reviewed, adapted and tested a wide variety of existing methods for neonatal chest sound separation for clean heart and lung sounds. Additionally, two methods were proposed, namely, NMF and NMCF with reference datasets of heart, lung and noise sounds. The evaluation results show, our proposed methods outperformed existing methods in both the artificial and real-world datasets with regards to SDR, SIR and SI-SDR, and signal quality, respectively. However, further work is still required for sound separation of respiratory support noise, as can be seen in the inferior results compared to existing methods for vital sign estimation. Possible directions are obtaining more clean samples either in the real-world setting or artificially and looking into deep learning methods.

E. Grooby thanks and acknowledges the support from the Monash Newborn team at Monash Children’s Hospital, Australia for data collection and Jinyuan He for the analysis of the newborn data in previous works.