All issues

This paper investigates neural network models and polynomial models based on Chebyshev polynomials for interference compensation. It is shown that the neural network model provides compensation for parasitic interference without the need for parameter tuning, unlike the polynomial model, which requires the selection of optimal delays. The L-BFGS method is applied to both architectures, achieving a compensation level comparable to the LS solution for the polynomial model, with an NMSE result of −23.59 dB and requiring fewer than 2000 iterations, confirming its high efficiency. Additionally, due to the strong generalization ability of neural network architectures, the first-order method for neural networks demonstrates faster convergence compared to the polynomial model. In 20 000 iterations, the neural network model achieves a 0.44 dB improvement in compensation level compared to the polynomial model. In contrast, the polynomial model can only achieve high compensation levels with optimal first-order method parameter tuning, highlighting one of the key advantages of neural network models.

Use of algorithms based on neural networks can be inefficient for small amounts of experimental data. Authors consider a solution of this problem in the context of modelling of properties of ceramic composite materials reinforced with carbon nanotubes using perceptron complex. This approach allowed us to obtain a mathematical description of the object of study with a minimal amount of input data (the amount of necessary experimental samples decreased 2–3.3 times). Authors considered different versions of perceptron complex structures. They found that the most appropriate structure has perceptron complex with breakthrough of two input variables. The relative error was only 6%. The selected perceptron complex was shown to be effective for predicting the properties of ceramic composites. The relative errors for output components were 0.3%, 4.2%, 0.4%, 2.9%, and 11.8%.

This article studies a method of constructing a predictive neural network model of a time series based on determining the composition of input variables, constructing a training sample and training itself using the back propagation method. Traditional methods of constructing predictive models of the time series are: the autoregressive model, the moving average model or the autoregressive model — the moving average allows us to approximate the time series by a linear dependence of the current value of the output variable on a number of its previous values. Such a limitation as linearity of dependence leads to significant errors in forecasting.

Mining Technologies using neural network modeling make it possible to approximate the time series by a nonlinear dependence. Moreover, the process of constructing of a neural network model (determining the composition of input variables, the number of layers and the number of neurons in the layers, choosing the activation functions of neurons, determining the optimal values of the neuron link weights) allows us to obtain a predictive model in the form of an analytical nonlinear dependence.

The determination of the composition of input variables of neural network models is one of the key points in the construction of neural network models in various application areas that affect its adequacy. The composition of the input variables is traditionally selected from some physical considerations or by the selection method. In this work it is proposed to use the behavior of the autocorrelation and private autocorrelation functions for the task of determining the composition of the input variables of the predictive neural network model of the time series.

In this work is proposed a method for determining the composition of input variables of neural network models for stationary and non-stationary time series, based on the construction and analysis of autocorrelation functions. Based on the proposed method in the Python programming environment are developed an algorithm and a program, determining the composition of the input variables of the predictive neural network model — the perceptron, as well as building the model itself. The proposed method was experimentally tested using the example of constructing a predictive neural network model of a time series that reflects energy consumption in different regions of the United States, openly published by PJM Interconnection LLC (PJM) — a regional network organization in the United States. This time series is non-stationary and is characterized by the presence of both a trend and seasonality. Prediction of the next values of the time series based on previous values and the constructed neural network model showed high approximation accuracy, which proves the effectiveness of the proposed method.

This paper presents the results of experimental validation of some structural issues concerning the practical use of methods to overcome catastrophic forgetting of neural networks. A comparison of current effective methods like EWC (Elastic Weight Consolidation) and WVA (Weight Velocity Attenuation) is made and their advantages and disadvantages are considered. It is shown that EWC is better for tasks where full retention of learned skills is required on all the tasks in the training queue, while WVA is more suitable for sequential tasks with very limited computational resources, or when reuse of representations and acceleration of learning from task to task is required rather than exact retention of the skills. The attenuation of the WVA method must be applied to the optimization step, i. e. to the increments of neural network weights, rather than to the loss function gradient itself, and this is true for any gradient optimization method except the simplest stochastic gradient descent (SGD). The choice of the optimal weights attenuation function between the hyperbolic function and the exponent is considered. It is shown that hyperbolic attenuation is preferable because, despite comparable quality at optimal values of the hyperparameter of the WVA method, it is more robust to hyperparameter deviations from the optimal value (this hyperparameter in the WVA method provides a balance between preservation of old skills and learning a new skill). Empirical observations are presented that support the hypothesis that the optimal value of this hyperparameter does not depend on the number of tasks in the sequential learning queue. And, consequently, this hyperparameter can be picked up on a small number of tasks and used on longer sequences.

We present results of a study aimed at identification of a controlled object’s channels based on postprocessing of measurements with development of a model of a multiple-input controlled object and subsequent active modelling experiment. The controlled object model is developed using approximation of its behavior by a neural network model using trends obtained during a passive experiment in the mode of normal operation. Recurrent neural network containing feedback elements allows to simulate behavior of dynamic objects; input and feedback time delays allow to simulate behavior of inertial objects with pure delay. The model was taught using examples of the object’s operation with a control system and is presented by a dynamic neural network and a model of a regulator with a known regulation function. The neural network model simulates the system’s behavior and is used to conduct active computing experiments. Neural network model allows to obtain the controlled object’s response to an exploratory stimulus, including a periodic one. The obtained complex frequency response is used to evaluate parameters of the object’s transfer system using the least squares method. We present an example of identification of a channel of the simulated control system. The simulated object has two input ports and one output port and varying transport delays in transfer channels. One of the input ports serves as a controlling stimulus, the second is a controlled perturbation. The controlled output value changes as a result of control stimulus produced by the regulator operating according to the proportional-integral regulation law based on deviation of the controlled value from the task. The obtained parameters of the object’s channels’ transfer functions are close to the parameters of the input simulated object. The obtained normalized error of the reaction for a single step-wise stimulus of the control system model developed based on identification of the simulated control system doesn’t exceed 0.08. The considered objects pertain to the class of technological processes with continuous production. Such objects are characteristic of chemical, metallurgic, mine-mill, pulp and paper, and other industries.

Solution of Dynamic Light Scattering problem makes it possible to determine particle size distribution (PSD) from the spectrum of the intensity of scattered light. As a result of experiment, an intensity curve is obtained. The experimentally obtained spectrum of intensity is compared with the theoretically expected spectrum, which is the Lorentzian line. The main task is to determine on the basis of these data the relative concentrations of particles of each class presented in the solution. The article presents a method for constructing and using a neural network trained on synthetic data to determine PSD in a solution in the range of 1–500 nm. The neural network has a fully connected layer of 60 neurons with the RELU activation function at the output, a layer of 45 neurons and the same activation function, a dropout layer and 2 layers with 15 and 1 neurons (network output). The article describes how the network has been trained and tested on synthetic and experimental data. On the synthetic data, the standard deviation metric (rmse) gave a value of 1.3157 nm. Experimental data were obtained for particle sizes of 200 nm, 400 nm and a solution with representatives of both sizes. The results of the neural network and the classical linear methods are compared. The disadvantages of the classical methods are that it is difficult to determine the degree of regularization: too much regularization leads to the particle size distribution curves are much smoothed out, and weak regularization gives oscillating curves and low reliability of the results. The paper shows that the neural network gives a good prediction for particles with a large size. For small sizes, the prediction is worse, but the error quickly decreases as the particle size increases.

The paper provides a method for selecting the composition of a refrigerant with a given isobaric cooling curve using an artificial neural network (ANN). This method is based on the use of 1D layers of a convolutional neural network. To train the neural network, we applied a technological model of a simple heat exchanger in the UniSim design program, using the Peng – Robinson equation of state.We created synthetic database on isobaric boiling curves of refrigerants of different compositions using the technological model. To record the database, an algorithm was developed in the Python programming language, and information on isobaric boiling curves for 1 049 500 compositions was uploaded using the COM interface. The compositions have generated by Monte Carlo method. Designed architecture of ANN allows select composition of a mixed refrigerant by 101 points of boiling curve. ANN gives mole flows of mixed refrigerant by composition (methane, ethane, propane, nitrogen) on the output layer. For training ANN, we used method of cyclical learning rate. For results demonstration we selected MR composition by natural gas cooling curve with a minimum temperature drop of 3 К and a maximum temperature drop of no more than 10 К, which turn better than we predicted via UniSim SQP optimizer and better than predicted by $k$-nearest neighbors algorithm. A significant value of this article is the fact that an artificial neural network can be used to select the optimal composition of the refrigerant when analyzing the cooling curve of natural gas. This method can help engineers select the composition of the mixed refrigerant in real time, which will help reduce the energy consumption of natural gas liquefaction.

In this paper we consider the controlled motion of a helical body with three blades in an ideal fluid, which is executed by rotating three internal rotors. We set the problem of selecting control actions, which ensure the motion of the body near the predetermined trajectory. To determine controls that guarantee motion near the given curve, we propose methods based on the application of hybrid genetic algorithms (genetic algorithms with real encoding and with additional learning of the leader of the population by a gradient method) and artificial neural networks. The correctness of the operation of the proposed numerical methods is estimated using previously obtained differential equations, which define the law of changing the control actions for the predetermined trajectory.

In the approach based on hybrid genetic algorithms, the initial problem of minimizing the integral functional reduces to minimizing the function of many variables. The given time interval is broken up into small elements, on each of which the control actions are approximated by Lagrangian polynomials of order 2 and 3. When appropriately adjusted, the hybrid genetic algorithms reproduce a solution close to exact. However, the cost of calculation of 1 second of the physical process is about 300 seconds of processor time.

To increase the speed of calculation of control actions, we propose an algorithm based on artificial neural networks. As the input signal the neural network takes the components of the required displacement vector. The node values of the Lagrangian polynomials which approximately describe the control actions return as output signals . The neural network is taught by the well-known back-propagation method. The learning sample is generated using the approach based on hybrid genetic algorithms. The calculation of 1 second of the physical process by means of the neural network requires about 0.004 seconds of processor time, that is, 6 orders faster than the hybrid genetic algorithm. The control calculated by means of the artificial neural network differs from exact control. However, in spite of this difference, it ensures that the predetermined trajectory is followed exactly.

In today’s technology-driven world, lite devices like Internet of Things (IoT) devices and microcontrollers (MCUs) are becoming increasingly common. These devices are more energyefficient and affordable, often with reduced features compared to the standard versions such as very limited memory and processing power for typical machine learning models. However, modern machine learning models can have millions of parameters, resulting in a large memory footprint. This complexity not only makes it difficult to deploy these large models on resource constrained devices but also increases the risk of latency and inefficiency in processing, which is crucial in some cases where real-time responses are required such as autonomous driving and medical diagnostics. In recent years, neural networks have seen significant advancements in model optimization techniques that help deployment and inference on these small devices. This narrative review offers a thorough examination of the progression and latest developments in neural network optimization, focusing on key areas such as quantization, pruning, knowledge distillation, and neural architecture search. It examines how these algorithmic solutions have progressed and how new approaches have improved upon the existing techniques making neural networks more efficient. This review is designed for machine learning researchers, practitioners, and engineers who may be unfamiliar with these methods but wish to explore the available techniques. It highlights ongoing research in optimizing networks for achieving better performance, lowering energy consumption, and enabling faster training times, all of which play an important role in the continued scalability of neural networks. Additionally, it identifies gaps in current research and provides a foundation for future studies, aiming to enhance the applicability and effectiveness of existing optimization strategies.

The key problems of remote forest pathology monitoring for coniferous forests affected by insect pests have been analyzed. It has been demonstrated that addressing these tasks requires the use of multiclass classification results for coniferous trees in high- and ultra-high-resolution images, which are promptly obtained through monitoring via satellites or unmanned aerial vehicles (UAVs). An analytical review of modern models and methods for multiclass classification of coniferous forest images was conducted, leading to the development of three fully convolutional neural network models: Mo-U-Net, At-Mo-U-Net, and Res-Mo-U-Net, all based on the classical U-Net architecture. Additionally, the Segformer transformer model was modified to suit the task. For RGB images of fir trees Abies sibirica affected by the four-eyed bark beetle Polygraphus proximus, captured using a UAV-mounted camera, two datasets were created: the first dataset contains image fragments and their corresponding reference segmentation masks sized 256 × 256 × 3 pixels, while the second dataset contains fragments sized 480 × 480 × 3 pixels. Comprehensive studies were conducted on each trained neural network model to evaluate both classification accuracy for assessing the degree of damage (health status) of Abies sibirica trees and computation speed using test datasets from each set. The results revealed that for fragments sized 256 × 256 × 3 pixels, the At-Mo-U-Net model with an attention mechanism is preferred alongside the Modified Segformer model. For fragments sized 480 × 480 × 3 pixels, the Res-Mo-U-Net hybrid model with residual blocks demonstrated superior performance. Based on classification accuracy and computation speed results for each developed model, it was concluded that, for production-scale multiclass classification of affected fir trees, the Res-Mo-U-Net model is the most suitable choice. This model strikes a balance between high classification accuracy and fast computation speed, meeting conflicting requirements effectively.