Dynamic Systems Modeling with Machine Learning

A time series is a finite or infinite sequence of elements that keep a specific chronological order with each other. These elements can be the product of a measurement process like the one mentioned above, or they can be generated in a completely synthetic way. In addition, depending on the characteristics of the measured variable, these elements could be represented in the space of one dimension or several dimensions. For example, if you want to analyze the motion behavior of a person, a time series will be obtained from the measurement of the acceleration variable, which by default has three components (one in each of the axes of motion), which gives as a result, a series of multi-dimensional time. If, on the contrary, it is desired to analyze the rectilinear motion of a particle, the most common will be to obtain a one-dimensional time series. From the point of view of the stored data, the difference in the representation of each element is that the uni-dimensional case will be given by a scalar, while the multi-dimensional case will be given by an ordered tuple of elements with vector structure.

The complexity of a series of time is directly related to the process that produces it and is usually associated with the variability of the process and the difficulty of finding a recognizable pattern in its behavior that can be used to make inferences about the process that originates it.

What is the objective of Dynamic Systems Modeling?

To meet this general objective, we have found it necessary to meet the following specific objectives:

- To study the viability of the representation of a dynamic system with characteristics extracted from a time series sampled from a single state variable utilizing a classical decomposition by the Wavelet Packet Transform.

- Develop a methodology based on Classic Signal Analysis and Machine Learning for the creation of predictive models of Dynamic Systems.

- Present a method for the generation of models of extraction of adaptive characteristics automatically that is capable of representing the dynamics of the system through Deep Learning techniques.

- To compare the previous proposal, experimentally with a set of methodologies for extracting characteristics.

- Develop a Dynamic Systems modeling method based on the reconstruction of the signal derived from a system variable.

- Analyze the feasibility of representing the dynamics of a system using another dynamic parameterized model.

- Present a methodology for the creation of generative models for detecting anomalies in Dynamic Systems through Deep Learning and Variation Inference.

- Evaluate the performance of all proposals in fault diagnosis applications, damage severity analysis, and/or fault detection in rotating machinery.

Learning time-series features:

 Here we focus on the analysis and modeling of time series with high variability. The perspective from which the problem is approached stems from the search for patterns through signal analysis and processing using a priori expert knowledge of the process to evolve into an unsupervised modeling methodology based solely on the data available from the measurements. To this end, three methods are proposed to address the problem of modeling a time series. The first proposal uses known signal processing techniques, which are selected based on the knowledge reported for the derivation of representative characteristics (with more notorious patterns) of the time series, and which will then be used for the construction of a classification model using the Random Forest Machine Learning model. 

The second proposal takes a step forward in the search for the independence of process knowledge, resulting in a method that assesses the possibility of using unsupervised feature drive with the use of a data-based learning model called Stacked Convolutional Auto encoder. It represents the time series in 2 dimensions, obtained with signal processing techniques, can extract useful features that will later be used to build a prediction model. These two initial approaches present the problem of needing time series that are labeled for the construction of the prediction model. Consequently, although the feature extraction process is unsupervised, as in the second case, under this approach, it is not possible to achieve a priori independence of knowledge in the data. 

Representation learning and one-class learning:

The third proposal is presented as a solution to this problem, where dependence on signal processing techniques is totally eliminated, and an Echo State Networks based approach is proposed to project the time series to a new representation space in an unsupervised manner. Under this approach, it is not possible to achieve a priori independence of knowledge in the data. 

The processes we will focus on to apply the above proposals belong to an area of ​​mechanical and process engineering called Condition-based Maintenance(CBM), which focuses on the study of techniques. They help to determine the general condition of a machine and/or its components to achieve (with good maintenance planning) extend its useful life and reduce operating costs. Within the CBM, the processes associated with the diagnosis of gears and rotating machinery bearings, which have been selected as the object of study of this work, are addressed. The objective that we seek for the task of the diagnosis of failures is to elaborate methodologies that allow identifying damages, and / or their severity, in gearboxes and bearings from only symptoms that can be captured from measurements in variables of the complete mechanical system.

Without a doubt, it is desirable to have fault classification models, and / or severity level,

These mechanical components have been chosen because they are the most common, important, and prone to failures in rotary machines, so we have both enough data for analysis and prior knowledge necessary to verify the robustness of the solutions obtained. The variable of the process from which the time series that we will use in our work is obtained is the vibration of the machine, which is obtained by the process of discretization of the signals captured by sensors called accelerometers. Although the time series with which we will work is a discrete representation of the actual signals, the two terms will have the same meaning throughout this work, unless explicitly stated otherwise.

Conclusion:

Based on the results obtained, both from a specific perspective to each of the proposals studied, and from a global perspective that combines and compares all these proposals. It also shows some possible future lines of research around the theme developed here, and the good results obtained, evaluating the modeling of dynamic systems with Machine Learning, hoping to motivate this path as one of the possible lines of work complementary to the rest of existing approaches.

Author: Vicki Lezama