All issues

In this work, we develop a model of the immune response to respiratory viral infections taking into account some particular properties of the SARS-CoV-2 infection. The model represents a system of ordinary differential equations for the concentrations of epithelial cells, immune cells, virus and inflammatory cytokines. Conventional analysis of the existence and stability of stationary points is completed by numerical simulations in order to study dynamics of solutions. Behavior of solutions is characterized by large peaks of virus concentration specific for acute respiratory viral infections.

At the first stage, we study the innate immune response based on the protective properties of interferon secreted by virus-infected cells. On the other hand, viral infection down-regulates interferon production. Their competition can lead to the bistability of the system with different regimes of infection progression with high or low intensity. In the case of infection outbreak, the incubation period and the maximal viral load depend on the initial viral load and the parameters of the immune response. In particular, increase of the initial viral load leads to shorter incubation period and higher maximal viral load.

In order to study the emergence and dynamics of cytokine storm, we consider proinflammatory cytokines produced by cells of the innate immune response. Depending on parameters of the model, the system can remain in the normal inflammatory state specific for viral infections or, due to positive feedback between inflammation and immune cells, pass to cytokine storm characterized by excessive production of proinflammatory cytokines. Furthermore, inflammatory cell death can stimulate transition to cytokine storm. However, it cannot sustain it by itself without the innate immune response. Assumptions of the model and obtained results are in qualitative agreement with the experimental and clinical data.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

An approximate mathematical model of blood flow in an axisymmetric blood vessel is studied. Such a vessel is understood as an infinitely long circular cylinder, the walls of which consist of elastic rings. Blood is considered as an incompressible fluid flowing in this cylinder. Increased pressure causes radially symmetrical stretching of the elastic rings. Following J. Lamb, the rings are located close to each other so that liquid does not flow between them. To mentally realize this, it is enough to assume that the rings are covered with an impenetrable film that does not have elastic properties. Only rings have elasticity. The considered model of blood flow in a blood vessel consists of three equations: the continuity equation, the law of conservation of momentum and the equation of state. An approximate procedure for reducing the equations under consideration to the Korteweg – de Vries (KdV) equation is considered, which was not fully considered by J. Lamb, only to establish the dependence of the coefficients of the KdV equation on the physical parameters of the considered model of incompressible fluid flow in an axisymmetric vessel. From the KdV equation, by a standard transition to traveling waves, ODEs of the third, second and first orders are obtained, respectively. Depending on the different cases of arrangement of the three stationary solutions of the first-order ODE, a cnoidal wave and a soliton are standardly obtained. The main attention is paid to an unbounded periodic solution, which we call a degenerate cnoidal wave. Mathematically, cnoidal waves are described by elliptic integrals with parameters defining amplitudes and periods. Soliton and degenerate cnoidal wave are described by elementary functions. The hemodynamic meaning of these types of decisions is indicated. Due to the fact that the sets of solutions to first-, second- and third-order ODEs do not coincide, it has been established that the Cauchy problem for second- and third-order ODEs can be specified at all points, and for first-order ODEs only at points of growth or decrease. The Cauchy problem for a first-order ODE cannot be specified at extremum points due to the violation of the Lipschitz condition. The degeneration of the cnoidal wave into a degenerate cnoidal wave, which can lead to rupture of the vessel walls, is numerically illustrated. The table below describes two modes of approach of a cnoidal wave to a degenerate cnoidal wave.

The biological structure is considered as an open nonequilibrium system which properties can be described on the basis of kinetic equations. New problems with nonequilibrium boundary conditions are introduced. The nonequilibrium distribution tends gradually to an equilibrium state. The region of spatial inhomogeneity has a scale depending on the rate of mass transfer in the open system and the characteristic time of metabolism. In the proposed approximation, the internal energy of the motion of molecules is much less than the energy of translational motion. Or in other terms we can state that the kinetic energy of the average blood velocity is substantially higher than the energy of chaotic motion of the same particles. We state that the relaxation problem models a living system. The flow of entropy to the system decreases in downstream, this corresponds to Shrödinger’s general ideas that the living system “feeds on” negentropy. We introduce a quantity that determines the complexity of the biosystem, more precisely, this is the difference between the nonequilibrium kinetic entropy and the equilibrium entropy at each spatial point integrated over the entire spatial region. Solutions to the problems of spatial relaxation allow us to estimate the size of biosystems as regions of nonequilibrium. The results are compared with empirical data, in particular, for mammals we conclude that the larger the size of animals, the smaller the specific energy of metabolism. This feature is reproduced in our model since the span of the nonequilibrium region is larger in the system where the reaction rate is shorter, or in terms of the kinetic approach, the longer the relaxation time of the interaction between the molecules. The approach is also used for estimation of a part of a living system, namely a green leaf. The problems of aging as degradation of an open nonequilibrium system are considered. The analogy is related to the structure, namely, for a closed system, the equilibrium of the structure is attained for the same molecules while in the open system, a transition occurs to the equilibrium of different particles, which change due to metabolism. Two essentially different time scales are distinguished, the ratio of which is approximately constant for various animal species. Under the assumption of the existence of these two time scales the kinetic equation splits in two equations, describing the metabolic (stationary) and “degradative” (nonstationary) parts of the process.

In this paper we consider a new modification of the discrete version of Mikhailov’s ‘power–society’ model, previously proposed by the author. This modification includes social-economical dynamics and corruption of the system similarly to continuous ‘power–society–economics–corruption’ model but is based on a stochastic cellular automaton describing the dynamics of power distribution in a hierarchy. This new version is founded on previously proposed ‘power–society’ system modeling cellular automaton, its cell state space enriched with variables corresponding to population, economic production, production assets volume and corruption level. The social-economical structure of the model is inherited from Solow and deterministic continuous ‘power–society–economics–corruption’ models. At the same time the new model is flexible, allowing to consider regional differentiation in all social and economical dynamics parameters, to use various production and demography models and to account for goods transit between the regions. A simulation system was built, including three power hierarchy levels, five regions and 100 municipalities. and a number of numerical experiments were carried out. This research yielded results showing specific changes of the dynamics in power distribution in hierarchy when corruption level increases. While corruption is zero (similar to the previous version of the model) the power distribution in hierarchy asymptotically tends to one of stationary states. If the corruption level increases substantially, volume of power in the system is subjected to irregular oscillations, and only much later tends to a stationary value. The meaning of these results can be interpreted as the fact that the stability of power hierarchy decreases when corruption level goes up.

To study nonlinear effects of biological species interactions numerical-analytical approach is being developed. The approach is based on the cosymmetry theory accounting for the phenomenon of the emergence of a continuous family of solutions to differential equations where each solution can be obtained from the appropriate initial state. In problems of mathematical ecology the onset of cosymmetry is usually connected with a number of relationships between the parameters of the system. When the relationships collapse families vanish, we get a finite number of isolated solutions instead of a continuum of solutions and transient process can be long-term, dynamics taking place in a neighborhood of a family that has vanished due to cosymmetry collapse.

We consider a model for spatiotemporal competition of predators or prey with an account for directed migration, Holling type II functional response and nonlinear prey growth function permitting Alley effect. We found out the conditions on system parameters under which there is linear with respect to population densities cosymmetry. It is demonstated that cosymmetry exists for any resource function in case of heterogeneous habitat. Numerical experiment in MATLAB is applied to compute steady states and oscillatory regimes in case of spatial heterogeneity.

The dynamics of three population interactions (two predators and a prey, two prey and a predator) are considered. The onset of families of stationary distributions and limit cycle branching out of equlibria of a family that lose stability are investigated in case of homogeneous habitat. The study of the system for two prey and a predator gave a wonderful result of species coexistence. We have found out parameter regions where three families of stable solutions can be realized: coexistence of two prey in absence of a predator, stationary and oscillatory distributions of three coexisting species. Cosymmetry collapse is analyzed and long-term transient dynamics leading to solutions with the exclusion of one of prey or extinction of a predator is established in the numerical experiment.

The paper deals with a prey-predator model, which describes the spatiotemporal dynamics of plankton community and the nutrients. The system is described by reaction-diffusion-advection equations in a onedimensional vertical column of water in the surface layer. Advective term of the predator equation represents the vertical movements of zooplankton with velocity, which is assumed to be proportional to the gradient of phytoplankton density. This study aimed to determine the conditions under which these movements (taxis) lead to the spatially heterogeneous structures generated by the system. Assuming diffusion coefficients of all model components to be equal the instability of the system in the vicinity of stationary homogeneous state with respect to small inhomogeneous perturbations is analyzed.

Necessary conditions for the flow-induced instability were obtained through linear stability analysis. Depending on the local kinetics parameters, increasing the taxis rate leads to Turing or wave instability. This fact is in good agreement with conditions for the emergence of spatial and spatiotemporal patterns in a minimal phytoplankton–zooplankton model after flow-induced instabilities derived by other authors. This mechanism of generating patchiness is more general than the Turing mechanism, which depends on strong conditions on the diffusion coefficients.

While the taxis exceeding a certain critical value, the wave number corresponding to the fastest growing mode remains unchanged. This value determines the type of spatial structure. In support of obtained results, the paper presents the spatiotemporal dynamics of the model components demonstrating Turing-type pattern and standing wave pattern.

The spatiotemporal dynamics of a three-component model for food web is considered. The model describes the interactions among resource, prey and predator that consumes both species. In a previous work, the author analyzed the model without taking into account spatial heterogeneity. This study continues the model study of the community considering the diffusion of individuals, as well as directed movements of the predator. It is assumed that the predator responds to the spatial change in the resource and prey density by occupying areas where species density is higher or avoiding them. Directed predator movement is described by the advection term, where velocity is proportional to the gradient of resource and prey density. The system is considered on a one-dimensional domain with zero-flux conditions as boundary ones. The spatiotemporal dynamics produced by model is determined by the system stability in the vicinity of stationary homogeneous state with respect to small inhomogeneous perturbations. The paper analyzes the possibility of wave instability leading to the emergence of autowaves and Turing instability, as a result of which stationary patterns are formed. Sufficient conditions for the existence of both types of instability are obtained. The influence of local kinetic parameters on the spatial structure formation was analyzed. It was shown that only Turing instability is possible when taxis on the resource is positive, but with a negative taxis, both types of instability are possible. The numerical solution of the system was found by using method of lines (MOL) with the numerical integration of ODE system by means of splitting techniques. The spatiotemporal dynamics of the system is presented in several variants, realizing one of the instability types. In the case of a positive taxis on the prey, both autowave and stationary structures are formed in smaller regions, with an increase in the region size, Turing structures are not formed. For negative taxis on the prey, stationary patterns is observed in both regions, while periodic structures appear only in larger areas.

In the development of the previously obtained result on modeling the dynamics of vegetation cover, due to variations in the temperature background, a new scheme for the interval analysis of the dynamics of floristic images of formations is presented in the case when the parameter of the response rate of the model of the dynamics of each counting plant species is set by the interval of scatter of its possible values. The detailed description of the functional parameters of macromodels of biodiversity, desired in fundamental research, taking into account the essential reasons for the observed evolutionary processes, may turn out to be a problematic task. The use of more reliable interval estimates of the variability of functional parameters “bypasses” the problem of uncertainty in the primary assessment of the evolution of the phyto-resource potential of the developed controlled territories. The solutions obtained preserve not only a qualitative picture of the dynamics of species diversity, but also give a rigorous, within the framework of the initial assumptions, a quantitative assessment of the degree of presence of each plant species. The practical significance of two-sided estimation schemes based on the construction of equations for the upper and lower boundaries of the trajectories of the scatter of solutions depends on the conditions and measure of proportional correspondence of the intervals of scatter of the initial parameters with the intervals of scatter of solutions. For dynamic systems, the desired proportionality is not always ensured. The given examples demonstrate the acceptable accuracy of interval estimation of evolutionary processes. It is important to note that the constructions of the estimating equations generate vanishing intervals of scatter of solutions for quasi-constant temperature perturbations of the system. In other words, the trajectories of stationary temperature states of the vegetation cover are not roughened by the proposed interval estimation scheme. The rigor of the result of interval estimation of the species composition of the vegetation cover of formations can become a determining factor when choosing a method in the problems of analyzing the dynamics of species diversity and the plant potential of territorial systems of resource-ecological monitoring. The possibilities of the proposed approach are illustrated by geoinformation images of the computational analysis of the dynamics of the vegetation cover of the Yamal Peninsula and by the graphs of the retro-perspective analysis of the floristic variability of the formations of the landscapelithological group “Upper” based on the data of the summer temperature background of the Salehard weather station from 2010 to 1935. The developed indicators of floristic variability and the given graphs characterize the dynamics of species diversity, both on average and individually in the form of intervals of possible states for each species of plant.