All issues

The paper presents a physico-mathematical model of the perturbed region formed in the lower D-layer of the ionosphere under the action of directed radio emission flux from a terrestrial stand of the megahertz frequency range, obtained as a result of comprehensive theoretical studies. The model is based on the consideration of a wide range of kinetic processes taking into account their nonequilibrium and in the two-temperature approximation for describing the transformation of the radio beam energy absorbed by electrons. The initial data on radio emission achieved by the most powerful radio-heating stands are taken in the paper. Their basic characteristics and principles of functioning, and features of the altitude distribution of the absorbed electromagnetic energy of the radio beam are briefly described. The paper presents the decisive role of the D-layer of the ionosphere in the absorption of the energy of the radio beam. On the basis of theoretical analysis, analytical expressions are obtained for the contribution of various inelastic processes to the distribution of the absorbed energy, which makes it possible to correctly describe the contribution of each of the processes considered. The work considers more than 60 components. The change of the component concentration describe about 160 reactions. All the reactions are divided into five groups according to their physical content: ionization-chemical block, excitation block of metastable electronic states, cluster block, excitation block of vibrational states and block of impurities. Blocks are interrelated and can be calculated both jointly and separately. The paper presents the behavior of the parameters of the perturbed region in daytime and nighttime conditions is significantly different at the same radio flux density: under day conditions, the maximum electron concentration and temperature are at an altitude of ~45–55 km; in night ~80 km, with the temperature of heavy particles rapidly increasing, which leads to the occurrence of a gas-dynamic flow. Therefore, a special numerical algorithm are developed to solve two basic problems: kinetic and gas dynamic. Based on the altitude and temporal behavior of concentrations and temperatures, the algorithm makes it possible to determine the ionization and emission of the ionosphere in the visible and infrared spectral range, which makes it possible to evaluate the influence of the perturbed region on radio engineering and optoelectronic devices used in space technology.

The review is based on some of the authors ’early works of particular scientific, methodological and practical interest and the greatest attention is paid to recent works, where quite detailed numerical studies of not only single, but also double and multiple explosions in a wide range of heights and environmental conditions have been performed . Since the shock wave of a powerful explosion is one of the main damaging factors in the lower atmosphere, the review focuses on both the physical analysis of their propagation and their interaction. Using the three-dimensional algorithms developed by the authors, the effects of interference and diffraction of several shock waves, which are interesting from a physical point of view, in the absence and presence of an underlying surface of various structures are considered. Quantitative characteristics are determined in the region of their maximum values, which is of known practical interest. For explosions in a dense atmosphere, some new analytical solutions based on the small perturbation method have been found that are convenient for approximate calculations. For a number of conditions, the possibility of using the self-similar properties of equations of the first and second kind to solve problems on the development of an explosion has been shown.

Based on numerical analysis, a fundamental change in the structure of the development of the perturbed region with a change in the height of the explosion in the range of 100–120 km is shown. At altitudes of more than 120 km, the geomagnetic field begins to influence the development of the explosion; therefore, even for a single explosion, the picture of the plasma flow after a few seconds becomes substantially three-dimensional. For the calculation of explosions at altitudes of 120–1000 km under the guidance of academician A. Kholodov. A special three-dimensional numerical algorithm based on the MHD approximation was developed. Numerous calculations were performed and for the first time a quite detailed picture of the three-dimensional flow of the explosion plasma was obtained with the formation of an upward jet in 5–10 s directed in the meridional plane approximately along the geomagnetic field. After some modification, this algorithm was used to calculate double explosions in the ionosphere, spaced a certain distance. The interaction between them was carried out both by plasma flows and through a geomagnetic field. Some results are given in this review and are described in detail in the original articles.

The paper presents a mathematical model that describes the process of obtaining biogas from livestock waste. This model describes the processes occurring in a biogas plant for mesophilic and thermophilic media, as well as for continuous and periodic modes of substrate inflow. The values of the coefficients of this model found earlier for the periodic mode, obtained by solving the problem of model identification from experimental data using a genetic algorithm, are given.

For the model of methanogenesis, an optimal control problem is formulated in the form of a Lagrange problem, whose criterial functionality is the output of biogas over a certain period of time. The controlling parameter of the task is the rate of substrate entry into the biogas plant. An algorithm for solving this problem is proposed, based on the numerical implementation of the Pontryagin maximum principle. In this case, a hybrid genetic algorithm with an additional search in the vicinity of the best solution using the method of conjugate gradients was used as an optimization method. This numerical method for solving an optimal control problem is universal and applicable to a wide class of mathematical models.

In the course of the study, various modes of submission of the substrate to the digesters, temperature environments and types of raw materials were analyzed. It is shown that the rate of biogas production in the continuous feed mode is 1.4–1.9 times higher in the mesophilic medium (1.9–3.2 in the thermophilic medium) than in the periodic mode over the period of complete fermentation, which is associated with a higher feed rate of the substrate and a greater concentration of nutrients in the substrate. However, the yield of biogas during the period of complete fermentation with a periodic mode is twice as high as the output over the period of a complete change of the substrate in the methane tank at a continuous mode, which means incomplete processing of the substrate in the second case. The rate of biogas formation for a thermophilic medium in continuous mode and the optimal rate of supply of raw materials is three times higher than for a mesophilic medium. Comparison of biogas output for various types of raw materials shows that the highest biogas output is observed for waste poultry farms, the least — for cattle farms waste, which is associated with the nutrient content in a unit of substrate of each type.

Theory of anomalous diffusion, which describe a vast number of transport processes with power law mean squared displacement, is actively advancing in recent years. Diffusion of liquids in porous media, carrier transport in amorphous semiconductors and molecular transport in viscous environments are widely known examples of anomalous deceleration of transport processes compared to the standard model.

Direct Monte Carlo simulation is a convenient tool for studying such processes. An efficient stochastic simulation algorithm is developed in the present paper. It is based on simple renewal process with interarrival times that have power law asymptotics. Analytical derivations show a deep connection between this class of random process and equations with fractional derivatives. The algorithm is further generalized by coupling it with chemical reaction simulation. It makes stochastic approach especially useful, because the exact form of integrodifferential evolution equations for reaction — subdiffusion systems is still a matter of debates.

Proposed algorithm relies on non-markovian random processes, hence one should carefully account for qualitatively new effects. The main question is how molecules leave the system during chemical reactions. An exact scheme which tracks all possible molecule combinations for every reaction channel is computationally infeasible because of the huge number of such combinations. It necessitates application of some simple heuristic procedures. Choosing one of these heuristics greatly affects obtained results, as illustrated by a series of numerical experiments.

We describe an engineering-psychological experiment that continues the study of ways to adapt a person to the increasing complexity of logical problems by presenting a series of problems of increasing complexity, which is determined by the volume of initial data. Tasks require calculations in an associative or non-associative system of operations. By the nature of the change in the time of solving the problem, depending on the number of necessary operations, we can conclude that a purely sequential method of solving problems or connecting additional brain resources to the solution in parallel mode. In a previously published experimental work, a person in the process of solving an associative problem recognized color images with meaningful images. In the new study, a similar problem is solved for abstract monochrome geometric shapes. Analysis of the result showed that for the second case, the probability of the subject switching to a parallel method of processing visual information is significantly reduced. The research method is based on presenting a person with two types of tasks. One type of problem contains associative calculations and allows a parallel solution algorithm. Another type of problem is the control one, which contains problems in which calculations are not associative and parallel algorithms are ineffective. The task of recognizing and searching for a given object is associative. A parallel strategy significantly speeds up the solution with relatively small additional resources. As a control series of problems (to separate parallel work from the acceleration of a sequential algorithm), we use, as in the previous experiment, a non-associative comparison problem in cyclic arithmetic, presented in the visual form of the game “rock, paper, scissors”. In this problem, the parallel algorithm requires a large number of processors with a small efficiency coefficient. Therefore, the transition of a person to a parallel algorithm for solving this problem is almost impossible, and the acceleration of processing input information is possible only by increasing the speed. Comparing the dependence of the solution time on the volume of source data for two types of problems allows us to identify four types of strategies for adapting to the increasing complexity of the problem: uniform sequential, accelerated sequential, parallel computing (where possible), or undefined (for this method) strategy. The Reducing of the number of subjects, who switch to a parallel strategy when encoding input information with formal images, shows the effectiveness of codes that cause subject associations. They increase the speed of human perception and processing of information. The article contains a preliminary mathematical model that explains this phenomenon. It is based on the appearance of a second set of initial data, which occurs in a person as a result of recognizing the depicted objects.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

The second part presents numerical studies of the parameters of the lower ionosphere at altitudes of 40–90 km when heated by powerful high-frequency radio waves of various frequencies and powers. The problem statement is considered in the first part of the article. The main attention is paid to the interrelation between the energy and kinetic parameters of the disturbed $D$-region of the ionosphere in the processes that determine the absorption and transformation of the radio beam energy flux in space and time. The possibility of a significant difference in the behavior of the parameters of the disturbed region in the daytime and at nighttime, both in magnitude and in space-time distribution, is shown. In the absence of sufficiently reliable values of the rate constants for a number of important kinetic processes, numerical studies were carried out in stages with the gradual addition of individual processes and kinetic blocks corresponding at the same time to a certain physical content. It is shown that the energy thresholds for inelastic collisions of electrons with air molecules are the main ones. This approach made it possible to detect the effect of the emergence of a self-oscillating mode of changing parameters if the main channel for energy losses in inelastic processes is the most energy-intensive process — ionization. This effect may play a role in plasma studies using high-frequency inductive and capacitive discharges. The results of calculations of the ionization and optical parameters of the disturbed $D$-region for daytime conditions are presented. The electron temperature, density, emission coefficients in the visible and infrared ranges of the spectrum are obtained for various values of the power of the radio beam and its frequency in the lower ionosphere. The height-time distribution of the absorbed radiation power is calculated, which is necessary in studies of higher layers of the ionosphere. The influence on the electron temperature and on the general behavior of the parameters of energy losses by electrons on the excitation of vibrational and metastable states of molecules has been studied in detail. It is shown that under nighttime conditions, when the electron concentration begins at altitudes of about 80 km, and the concentration of heavy particles decreases by two orders of magnitude compared to the average $D$-region, large-scale gas-dynamic motion can develop with sufficient radio emission power The algorithm was developed based on the McCormack method and two-dimensional gas-dynamic calculations of the behavior of the parameters of the perturbed region were performed with some simplifications of the kinetics.

Modeling of channel processes in the study of coastal channel deformations requires the calculation of hydrodynamic flow parameters that take into account the existence of secondary transverse currents formed at channel curvature. Three-dimensional modeling of such processes is currently possible only for small model channels; for real river flows, reduced-dimensional models are needed. At the same time, the reduction of the problem from a three-dimensional model of the river flow movement to a two-dimensional flow model in the cross-section assumes that the hydrodynamic flow under consideration is quasi-stationary and the hypotheses about the asymptotic behavior of the flow along the flow coordinate of the cross-section are fulfilled for it. Taking into account these restrictions, a mathematical model of the problem of the a stationary turbulent calm river flow movement in a channel cross-section is formulated. The problem is formulated in a mixed formulation of velocity — “vortex – stream function”. As additional conditions for problem reducing, it is necessary to specify boundary conditions on the flow free surface for the velocity field, determined in the normal and tangential direction to the cross-section axis. It is assumed that the values of these velocities should be determined from the solution of auxiliary problems or obtained from field or experimental measurement data.

To solve the formulated problem, the finite element method in the Petrov – Galerkin formulation is used. Discrete analogue of the problem is obtained and an algorithm for solving it is proposed. Numerical studies have shown that, in general, the results obtained are in good agreement with known experimental data. The authors associate the obtained errors with the need to more accurately determine the circulation velocities field at crosssection of the flow by selecting and calibrating a more appropriate model for calculating turbulent viscosity and boundary conditions at the free boundary of the cross-section.

We consider two possible computational methods of the interpretation of experimental data obtained by means of the magnetic force microscopy. These methods of macrospin distribution simulation and reconstruction can be used for research of magnetization reversal processes of nanodots in ordered 2D arrays of nanodots. New approaches to the development of high-performance superscale algorithms for parallel executing on a supercomputer clusters for solving direct and inverse task of the modeling of magnetic states, types of ordering, reversal processes of nanosystems with a collective behavior are proposed. The simulation results are consistent with experimental results.

Investigation of influence of non-equilibrium initial states and structural disorder on characteristics of anomalous slow non-equilibrium critical behavior of three-dimensional Ising model is carried out. The unique ageing properties and violations of the equilibrium fluctuation-dissipation theorem are observed for considered pure and disordered systems which were prepared in high-temperature initial state and then quenched in their critical points. The heat-bath algorithm description of ageing properties in non-equilibrium critical behavior of three-dimensional Ising model with spin concentrations p = 1.0, p = 0.8, and 0.6 is realized. On the base of analysis of such two-time quantities as autocorrelation function and dynamical susceptibility were demonstrated the ageing effects and were calculated asymptotic values of universal fluctuation-dissipation ratio in these systems. It was shown that the presence of defects leads to aging gain.