Transformer-Based Denoising of Mechanical Vibration Signals for System Health Monitoring

In today’s interconnected world, the importance of robust and responsive systems cannot be overstated. Whether in the context of industrial manufacturing, healthcare, transportation, or information technology, the reliance on complex systems to maintain operational continuity is ubiquitous [1, 2]. System health monitoring (SHM) has emerged as a critical field that ensures the smooth functioning of these complex mechanisms, mitigating the risk of sudden failure, and thus protecting the intricate web of daily operations across various sectors.

SHM is a multifaceted discipline that draws on engineering, data analytics, predictive modeling, and other advanced technologies to track the performance of a system in real-time. Its primary aim is to detect, diagnose, and predict anomalies or failures in a system, allowing for timely intervention and maintenance [3]. This is not only crucial for minimizing downtime but also for optimizing the lifecycle and efficiency of the system. System health monitoring is akin to a vigilant sentinel, continually assessing various parameters to maintain optimal performance and safety.

The evolution of SHM has been rapid, paralleling advancements in sensor technology, machine learning, and data processing. It is now possible to integrate a multitude of sensors that can track everything from temperature, pressure, vibration, to more nuanced indicators of system performance [4]. These data streams are then processed using intelligent algorithms that can discern patterns indicative of a potential failure or inefficiency. The application of artificial intelligence and machine learning has particularly transformed SHM, enabling predictive analytics that can forecast failures well before they become critical, allowing for preventive measures that can save both time and resources. The generic signal processing methodology enhances the analysis of acoustic signals with scattered sound field [5]. However, the implementation of SHM is not without challenges. The complexity of modern systems, coupled with the ever-growing demands for efficiency and sustainability, requires a nuanced understanding of the particular system’s architecture and operational context. The integration of sensors and analytical tools must be carefully tailored to the specific needs and constraints of the system in question. Furthermore, ethical considerations related to data privacy and security must be judiciously addressed to maintain trust and compliance with regulatory requirements.

In industries where system failures can result in catastrophic consequences, such as aviation, nuclear energy, and healthcare, SHM is not merely an optimization tool but a vital component for safety and compliance [6]. Even in less critical domains, the economic impact of unplanned downtime can be substantial, underscoring the universal relevance of SHM. Vibration signals have been studied to detect the artifects of mechanical products [7]. This paper aims to explore the contemporary landscape of system health monitoring, delving into the underlying technologies, methodologies, applications, and challenges. Through a comprehensive examination of recent developments and case studies, it will illuminate the vital role that SHM plays in our modern world, fostering a deeper understanding of its potential to drive innovation, efficiency, and safety across various domains.

In mechanical systems, the reliability and effectiveness of data analysis and control depend heavily on the quality of the acquired signals. Noise interference, stemming from various sources, often corrupts these signals and hampers accurate monitoring and control. Signal denoising, the process of extracting the true signal from the noise-infected observation, plays a pivotal role in improving the efficiency and reliability of mechanical systems. In industrial applications, transportation, healthcare equipment, and more, denoising is indispensable for accurate system modeling, fault detection, and performance enhancement [8].

The challenge of signal denoising lies in the diversity of noise types and their often complex interaction with the underlying true signal. Mechanical systems are typically subjected to various environmental noises, equipment vibrations, electronic interference, and human-induced errors [9]. The task of distinguishing the genuine signal from this multifaceted noise without compromising essential information is a non-trivial problem that requires sophisticated techniques and algorithms [10].

This paper aims to survey the field of signal denoising with a specific focus on its applications in mechanical systems. It will delve into the theoretical foundations, methodologies, and state-of-the-art technologies used for denoising, with a particular emphasis on exploring how these methods can be adapted to meet the unique demands of different mechanical systems.

With the advent of wavelet transform in the late 1980s, a new era of signal denoising began. Donoho and Johnstone’s seminal work in 1994 introduced wavelet thresholding, providing a powerful tool to preserve signal characteristics while removing noise [11, 12]. This approach revolutionized denoising applications in mechanical systems, allowing for more nuanced handling of non-stationary signals typical in such contexts. Recent advances integrating wavelet transform and Autoencoder shed light on 2D signal denoising, achieving state of the art denoising performance with minimal amount of parameters [13]. The field of signal denoising has a rich and diverse history, spanning several decades. In the early stages, linear filtering techniques, such as the Wiener filter, were commonly used. These methods provided a simple way to reduce noise but often led to the loss of important signal features [14].

The development of adaptive filtering techniques, like the Kalman filter, added to the arsenal of tools for denoising, offering more flexibility in handling diverse noise structures. More recently, the emergence of machine learning, particularly deep learning techniques like Convolutional Neural Networks (CNNs), has opened up new frontiers in denoising [15]. These methods leverage large datasets to learn intricate noise patterns, providing highly customized denoising solutions [8].

However, despite these advancements, challenges remain in the application of denoising to mechanical systems. The complexity of noises, non-linearity of mechanical systems, and the need for real-time processing pose persistent hurdles. Recent research has begun to explore hybrid methods, combining traditional statistical techniques with machine learning, to create more robust and adaptive denoising frameworks[14, 14].

This paper will provide an in-depth examination of these historical developments, current state-of-the-art methodologies, and the emerging trends in signal denoising for mechanical systems. By synthesizing insights from across this broad spectrum, it seeks to identify potential future directions and innovations that can further enhance the efficiency and reliability of mechanical systems through advanced signal denoising techniques.

In this work, we have presented a novel approach to mechanical vibration signal denoising utilizing a transformer-based architecture. The complexity and characteristics of mechanical vibrations necessitate a robust and adaptive method capable of discerning relevant patterns while eliminating noise. Our model, designed with a sequence length of 128 and an embedding size of 64, is particularly tailored to these requirements.

The innovative use of a Multi-Head Attention layer with 8 attention heads has enabled the model to simultaneously consider various aspects of the signal sequence, thereby enhancing its ability to recognize and extract essential features. The attention mechanism, paired with the Feed-Forward Neural Network and Residual Connections, provides flexibility and depth, facilitating the learning of intricate signal relationships.

Further stability in training has been achieved through Layer Normalization, applied after both the attention and feed-forward layers, ensuring numerical stability and efficient convergence. The training process, guided by the Mean Squared Error loss function and optimized using the Adam optimizer, was carried out for 50 epochs, with a carefully selected batch size of 32 and a validation split of 20%.

The final model has demonstrated effectiveness in denoising mechanical vibration signals, retaining crucial information while filtering out undesired noise. The architecture’s specific design, coupled with the thoughtful selection of parameters, has resulted in a method capable of adapting to the complex nature of mechanical systems.

Future work may explore further optimization of the architecture and parameters, integration with other denoising techniques, and extensive testing across various types of mechanical systems. The current results, however, lay a strong foundation for applying deep learning transformers in the critical field of mechanical vibration signal analysis and denoising, with promising implications for industrial maintenance, monitoring, and diagnostics.