All issues

The mathematical model, finite-difference schemes and algorithms for computation of transient thermoand hydrodynamic processes involved in commissioning the unified system including the oil producing well, electrical submersible pump and fractured-porous reservoir with bottom water are developed. These models are implemented in the computer package to simulate transient processes with simultaneous visualization of their results along with computations. An important feature of the package Oil-RWP is its interaction with the special external program GCS which simulates the work of the surface electric control station and data exchange between these two programs. The package Oil-RWP sends telemetry data and current parameters of the operating submersible unit to the program module GCS (direct coupling). The station controller analyzes incoming data and generates the required control parameters for the submersible pump. These parameters are sent to Oil-RWP (feedback). Such an approach allows us to consider the developed software as the “Intellectual Well System”.

Some principal results of the simulations can be briefly presented as follows. The transient time between inaction and quasi-steady operation of the producing well depends on the well stream watering, filtration and capacitive parameters of oil reservoir, physical-chemical properties of phases and technical characteristics of the submersible unit. For the large time solution of the nonstationary equations governing the nonsteady processes is practically identical to the inverse quasi-stationary problem solution with the same initial data. The developed software package is an effective tool for analysis, forecast and optimization of the exploiting parameters of the unified oil-producing complex during its commissioning into the operating regime.

The work is devoted to the description of the author’s formal statements of the clustering problem for a given number of clusters, algorithms for their solution, as well as the results of using this toolkit in medicine.

The solution of the formulated problems by exact algorithms of implementations of even relatively low dimensions before proving optimality is impossible in a finite time due to their belonging to the NP class.

In this regard, we have proposed a hybrid algorithm that combines the advantages of precise methods based on clustering in paired distances at the initial stage with the speed of methods for solving simplified problems of splitting by cluster centers at the final stage. In the development of this direction, a sequential hybrid clustering algorithm using random search in the paradigm of swarm intelligence has been developed. The article describes it and presents the results of calculations of applied clustering problems.

To determine the effectiveness of the developed tools for optimal clustering of multidimensional objects according to a variety of heterogeneous indicators, a number of computational experiments were performed using data sets including socio-demographic, clinical anamnestic, electroencephalographic and psychometric data on the cognitive status of patients of the cardiology clinic. An experimental proof of the effectiveness of using local search algorithms in the paradigm of swarm intelligence within the framework of a hybrid algorithm for solving optimal clustering problems has been obtained.

The results of the calculations indicate the actual resolution of the main problem of using the discrete optimization apparatus — limiting the available dimensions of task implementations. We have shown that this problem is eliminated while maintaining an acceptable proximity of the clustering results to the optimal ones. The applied significance of the obtained clustering results is also due to the fact that the developed optimal clustering toolkit is supplemented by an assessment of the stability of the formed clusters, which allows for known factors (the presence of stenosis or older age) to additionally identify those patients whose cognitive resources are insufficient to overcome the influence of surgical anesthesia, as a result of which there is a unidirectional effect of postoperative deterioration of complex visual-motor reaction, attention and memory. This effect indicates the possibility of differentiating the classification of patients using the proposed tools.

The program for automatic revealing of threshold values for characterizing physiological state of vessels and detection of early stages of retina pathology is offered. The algorithm is based on checking character of crossing sites of vessel images with the "mask" consisting of concentric circumferences (the first circumference is imposed directly on the sclera capsules of an optic nerve disk). The new method allows revealing of a network of blood vessels and flanking zones and detection of initial stage of pathological changes in a retina by digital images.

In this paper, we propose an adaptive algorithm that simulates the process of forming the initial behavioral skills on the example of the system ‘eye-arm’ animat. The situation is the formation of the initial behavioral skills occurs, for example, when a child masters the management of their hands by understanding the relationship between baseline unidentified spots on the retina of his eye and the position of the real object. Since the body control skills are not ‘hardcoded’ initially in the brain and the spinal cord at the level of instincts, the human child, like most young of other mammals, it is necessary to develop these skills in search behavior mode. Exploratory behavior begins with trial and error and then its contribution is gradually reduced as the development of the body and its environment. Since the correct behavior patterns at this stage of development of the organism does not exist for now, then the only way to select the right skills is a positive reinforcement to achieve the objective. A key feature of the proposed algorithm is to fix in the imprinting mode, only the final action that led to success, and that is very important, led to the familiar imprinted situation clearly leads to success. Over time, the continuous chain is lengthened right action — maximum use of previous positive experiences and negative ‘forgotten’ and not used.

Thus there is the gradual replacement of the random search purposeful actions that observed in the real young. Thus, the algorithm is able to establish a correspondence between the laws of the world and the ‘inner feelings’, the internal state of the animat. The proposed animat model was used 2 types of neural networks: 1) neural network NET1 to the input current which is fed to the position of the brush arms and the target point, and the output of motor commands, directing ‘brush’ manipulator animat to the target point; 2) neural network NET2 is received at the input of target coordinates and the current coordinates of the ‘brush’ and the output value is formed likelihood that the animat already ‘know’ this situation, and he ‘knows’ how to react to it. With this architecture at the animat has to rely on the ‘experience’ of neural networks to recognize situations where the response from NET2 network of close to 1, and on the other hand, run a random search, when the experience of functioning in this area of the visual field in animat not (response NET2 close to 0).

The paper presents a physico-mathematical model of the perturbed region formed in the lower D-layer of the ionosphere under the action of directed radio emission flux from a terrestrial stand of the megahertz frequency range, obtained as a result of comprehensive theoretical studies. The model is based on the consideration of a wide range of kinetic processes taking into account their nonequilibrium and in the two-temperature approximation for describing the transformation of the radio beam energy absorbed by electrons. The initial data on radio emission achieved by the most powerful radio-heating stands are taken in the paper. Their basic characteristics and principles of functioning, and features of the altitude distribution of the absorbed electromagnetic energy of the radio beam are briefly described. The paper presents the decisive role of the D-layer of the ionosphere in the absorption of the energy of the radio beam. On the basis of theoretical analysis, analytical expressions are obtained for the contribution of various inelastic processes to the distribution of the absorbed energy, which makes it possible to correctly describe the contribution of each of the processes considered. The work considers more than 60 components. The change of the component concentration describe about 160 reactions. All the reactions are divided into five groups according to their physical content: ionization-chemical block, excitation block of metastable electronic states, cluster block, excitation block of vibrational states and block of impurities. Blocks are interrelated and can be calculated both jointly and separately. The paper presents the behavior of the parameters of the perturbed region in daytime and nighttime conditions is significantly different at the same radio flux density: under day conditions, the maximum electron concentration and temperature are at an altitude of ~45–55 km; in night ~80 km, with the temperature of heavy particles rapidly increasing, which leads to the occurrence of a gas-dynamic flow. Therefore, a special numerical algorithm are developed to solve two basic problems: kinetic and gas dynamic. Based on the altitude and temporal behavior of concentrations and temperatures, the algorithm makes it possible to determine the ionization and emission of the ionosphere in the visible and infrared spectral range, which makes it possible to evaluate the influence of the perturbed region on radio engineering and optoelectronic devices used in space technology.

The review is based on some of the authors ’early works of particular scientific, methodological and practical interest and the greatest attention is paid to recent works, where quite detailed numerical studies of not only single, but also double and multiple explosions in a wide range of heights and environmental conditions have been performed . Since the shock wave of a powerful explosion is one of the main damaging factors in the lower atmosphere, the review focuses on both the physical analysis of their propagation and their interaction. Using the three-dimensional algorithms developed by the authors, the effects of interference and diffraction of several shock waves, which are interesting from a physical point of view, in the absence and presence of an underlying surface of various structures are considered. Quantitative characteristics are determined in the region of their maximum values, which is of known practical interest. For explosions in a dense atmosphere, some new analytical solutions based on the small perturbation method have been found that are convenient for approximate calculations. For a number of conditions, the possibility of using the self-similar properties of equations of the first and second kind to solve problems on the development of an explosion has been shown.

Based on numerical analysis, a fundamental change in the structure of the development of the perturbed region with a change in the height of the explosion in the range of 100–120 km is shown. At altitudes of more than 120 km, the geomagnetic field begins to influence the development of the explosion; therefore, even for a single explosion, the picture of the plasma flow after a few seconds becomes substantially three-dimensional. For the calculation of explosions at altitudes of 120–1000 km under the guidance of academician A. Kholodov. A special three-dimensional numerical algorithm based on the MHD approximation was developed. Numerous calculations were performed and for the first time a quite detailed picture of the three-dimensional flow of the explosion plasma was obtained with the formation of an upward jet in 5–10 s directed in the meridional plane approximately along the geomagnetic field. After some modification, this algorithm was used to calculate double explosions in the ionosphere, spaced a certain distance. The interaction between them was carried out both by plasma flows and through a geomagnetic field. Some results are given in this review and are described in detail in the original articles.

Theory of anomalous diffusion, which describe a vast number of transport processes with power law mean squared displacement, is actively advancing in recent years. Diffusion of liquids in porous media, carrier transport in amorphous semiconductors and molecular transport in viscous environments are widely known examples of anomalous deceleration of transport processes compared to the standard model.

Direct Monte Carlo simulation is a convenient tool for studying such processes. An efficient stochastic simulation algorithm is developed in the present paper. It is based on simple renewal process with interarrival times that have power law asymptotics. Analytical derivations show a deep connection between this class of random process and equations with fractional derivatives. The algorithm is further generalized by coupling it with chemical reaction simulation. It makes stochastic approach especially useful, because the exact form of integrodifferential evolution equations for reaction — subdiffusion systems is still a matter of debates.

Proposed algorithm relies on non-markovian random processes, hence one should carefully account for qualitatively new effects. The main question is how molecules leave the system during chemical reactions. An exact scheme which tracks all possible molecule combinations for every reaction channel is computationally infeasible because of the huge number of such combinations. It necessitates application of some simple heuristic procedures. Choosing one of these heuristics greatly affects obtained results, as illustrated by a series of numerical experiments.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

We consider two possible computational methods of the interpretation of experimental data obtained by means of the magnetic force microscopy. These methods of macrospin distribution simulation and reconstruction can be used for research of magnetization reversal processes of nanodots in ordered 2D arrays of nanodots. New approaches to the development of high-performance superscale algorithms for parallel executing on a supercomputer clusters for solving direct and inverse task of the modeling of magnetic states, types of ordering, reversal processes of nanosystems with a collective behavior are proposed. The simulation results are consistent with experimental results.

The paper contains numerical simulation of nonisothermal nonlinear flow in a porous medium. Twodimensional unsteady problem of heavy oil, water and steam flow is considered. Oil phase consists of two pseudocomponents: light and heavy fractions, which like the water component, can vaporize. Oil exhibits viscoplastic rheology, its filtration does not obey Darcy's classical linear law. Simulation considers not only the dependence of fluids density and viscosity on temperature, but also improvement of oil rheological properties with temperature increasing.

To solve this problem numerically we use streamline method with splitting by physical processes, which consists in separating the convective heat transfer directed along filtration from thermal conductivity and gravitation. The article proposes a new approach to streamline methods application, which allows correctly simulate nonlinear flow problems with temperature-dependent rheology. The core of this algorithm is to consider the integration process as a set of quasi-equilibrium states that are results of solving system on a global grid. Between these states system solved on a streamline grid. Usage of the streamline method allows not only to accelerate calculations, but also to obtain a physically reliable solution, since integration takes place on a grid that coincides with the fluid flow direction.

In addition to the streamline method, the paper presents an algorithm for nonsmooth coefficients accounting, which arise during simulation of viscoplastic oil flow. Applying this algorithm allows keeping sufficiently large time steps and does not change the physical structure of the solution.

Obtained results are compared with known analytical solutions, as well as with the results of commercial package simulation. The analysis of convergence tests on the number of streamlines, as well as on different streamlines grids, justifies the applicability of the proposed algorithm. In addition, the reduction of calculation time in comparison with traditional methods demonstrates practical significance of the approach.