All issues

Seasonal migrations and herd instinct are traditionally recognized as wild reindeer (Rangifer tarandus L.) species-specific behavioral signs. These animals are forced to overcome water obstacles during the migrations. Behaviour peculiarities are considered as the result of the selection process, which has chosen among the sets of strategies, as the only evolutionarily stable one, determining the reproduction and biological survival of wild reindeer as a species. Natural processes in the Taimyr population wild reindeer are currently occurring against the background of an increase in the influence of negative factors due to the escalation of the industrial development of the Arctic. That is why the need to identify the ethological features of these animals completely arose. This paper presents the results of applying the classical methods of the theory of optimal control and differential games to the wild reindeer study of the migration patterns in overcoming water barriers, including major rivers. Based on these animals’ ethological features and behavior forms, the herd is presented as a controlled dynamic system, which presents also two classes of individuals: the leader and the rest of the herd, for which their models, describing the trajectories of their movement, are constructed. The models are based on hypotheses, which are the mathematical formalization of some animal behavior patterns. This approach made it possible to find the trajectory of the important one using the methods of the optimal control theory, and in constructing the trajectories of other individuals, apply the principle of control with a guide. Approbation of the obtained results, which can be used in the formation of a common “platform” for the adaptive behavior models systematic construction and as a reserve for the cognitive evolution models fundamental development, is numerically carried out using a model example with observational data on the Werchnyaya Taimyra River.

The article deals with the problem of a distributed system operation reliability. The system core is an open integration platform that provides interaction of varied software for modeling gas transportation. Some of them provide an access through thin clients on the cloud technology “software as a service”. Mathematical models of operation, transmission and computing are to ensure the operation of an automated dispatching system for oil and gas transportation. The paper presents a system solution based on the theory of Markov random processes and considers the stable operation stage. The stationary operation mode of the Markov chain with continuous time and discrete states is described by a system of Chapman–Kolmogorov equations with respect to the average numbers (mathematical expectations) of the objects in certain states. The objects of research are both system elements that are present in a large number – thin clients and computing modules, and individual ones – a server, a network manager (message broker). Together, they are interacting Markov random processes. The interaction is determined by the fact that the transition probabilities in one group of elements depend on the average numbers of other elements groups.

The authors propose a multi-criteria dispersion model of risk assessment for such systems (both in the broad and narrow sense, in accordance with the IEC standard). The risk is the standard deviation of estimated object parameter from its average value. The dispersion risk model makes possible to define optimality criteria and whole system functioning risks. In particular, for a thin client, the following is calculated: the loss profit risk, the total risk of losses due to non-productive element states, and the total risk of all system states losses.

Finally the paper proposes compromise schemes for solving the multi-criteria problem of choosing the optimal operation strategy based on the selected set of compromise criteria.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

The paper proposes a mathematical model for the therapy of glioma, taking into account the blood-brain barrier, radiotherapy and antibody therapy. The parameters were estimated from experimental data and the evaluation of the effect of parameter values on the effectiveness of treatment and the prognosis of the disease were obtained. The possible variants of sequential use of radiotherapy and the effect of antibodies have been explored. The combined use of radiotherapy with intravenous administration of $mab$ $Cx43$ leads to a potentiation of the therapeutic effect in glioma.

Radiotherapy must precede chemotherapy, as radio exposure reduces the barrier function of endothelial cells. Endothelial cells of the brain vessels fit tightly to each other. Between their walls are formed so-called tight contacts, whose role in the provision of BBB is that they prevent the penetration into the brain tissue of various undesirable substances from the bloodstream. Dense contacts between endothelial cells block the intercellular passive transport.

The mathematical model consists of a continuous part and a discrete one. Experimental data on the volume of glioma show the following interesting dynamics: after cessation of radio exposure, tumor growth does not resume immediately, but there is some time interval during which glioma does not grow. Glioma cells are divided into two groups. The first group is living cells that divide as fast as possible. The second group is cells affected by radiation. As a measure of the health of the blood-brain barrier system, the ratios of the number of BBB cells at the current moment to the number of cells at rest, that is, on average healthy state, are chosen.

The continuous part of the model includes a description of the division of both types of glioma cells, the recovery of BBB cells, and the dynamics of the drug. Reducing the number of well-functioning BBB cells facilitates the penetration of the drug to brain cells, that is, enhances the action of the drug. At the same time, the rate of division of glioma cells does not increase, since it is limited not by the deficiency of nutrients available to cells, but by the internal mechanisms of the cell. The discrete part of the mathematical model includes the operator of radio interaction, which is applied to the indicator of BBB and to glial cells.

Within the framework of the mathematical model of treatment of a cancer tumor (glioma), the problem of optimal control with phase constraints is solved. The patient’s condition is described by two variables: the volume of the tumor and the condition of the BBB. The phase constraints delineate a certain area in the space of these indicators, which we call the survival area. Our task is to find such treatment strategies that minimize the time of treatment, maximize the patient’s rest time, and at the same time allow state indicators not to exceed the permitted limits. Since the task of survival is to maximize the patient’s lifespan, it is precisely such treatment strategies that return the indicators to their original position (and we see periodic trajectories on the graphs). Periodic trajectories indicate that the deadly disease is translated into a chronic one.

At the middle of the 2000-th, scientists studying the functioning of insect communities identified four basic patterns of the organizational structure of such communities. (i) Cooperation is more developed in groups with strong kinship. (ii) Cooperation in species with large colony sizes is often more developed than in species with small colony sizes. And small-sized colonies often exhibit greater internal reproductive conflict and less morphological and behavioral specialization. (iii) Within a single species, brood size (i. e., in a sense, efficiency) per capita usually decreases as colony size increases. (iv) Advanced cooperation tends to occur when resources are limited and intergroup competition is fierce. Thinking of the functioning of a group of organisms as a two-level competitive market in which individuals face the problem of allocating their energy between investment in intergroup competition and investment in intragroup competition, i. e., an internal struggle for the share of resources obtained through intergroup competition, we can compare such a biological situation with the economic phenomenon of “coopetition” — the cooperation of competing agents with the goal of later competitively dividing the resources won in consequence In the framework of economic researches the effects similar to (ii) — in the framework of large and small group competition the optimal strategy of large group would be complete squeezing out of the second group and monopolization of the market (i. e. large groups tend to act cooperatively) and (iii) — there are conditions, in which the size of the group has a negative impact on productivity of each of its individuals (this effect is called the paradox of group size or Ringelman effect). The general idea of modeling such effects is the idea of proportionality — each individual (an individual/rational agent) decides what share of his forces to invest in intergroup competition and what share to invest in intragroup competition. The group’s gain must be proportional to its total investment in competition, while the individual’s gain is proportional to its contribution to intra-group competition. Despite the prevalence of empirical observations, no gametheoretic model has yet been introduced in which the empirically observed effects can be confirmed. This paper proposes a model that eliminates the problems of previously existing ones and the simulation of Nash equilibrium states within the proposed model allows the above effects to be observed in numerical experiments.

Based on the time-discrete model, we study the effect of selective proportional harvesting on the population dynamics with age and sex structure. When constructing the model, we assume that the population birth rate depends on the ratio of the sexes and the number of formed pairs. The regulation of population growth is carried out by limiting the juvenile’s survival when the survival of immature individuals decreases with an increase in the numbers of sex and age classes. We consider cases where the harvest is carried out only from a younger age class or from a group of mature females or males. We find that the harvesting of males or females at the optimal level is responsible for changing the ratio of females to males (taking into account the average size of the harem). We show that the maximum number of harvested males is achieved either at such a harvest rate when their excess number is withdrawn and the balance of sexes is established or at such an optimal catch quota at which the sex ratio is shifted towards breeding females. Optimal female harvesting, in which the highest number of them are taken, either maintains a preexisting shortage of adult males or leads to an excess of males or the fixing of a sex balance. We find that, depending on the population parameters for all considered harvesting strategies, the hydra effect can observe, i. e., the equilibrium size of the exploited sex and age-specific group (after reproduction) can increase with the growth of harvesting intensity. The selective harvesting, due to which the hydra effect occurs, simultaneously leads to an increase remaining population size and the number of harvested individuals. At the same time, the size of the exploited group after reproduction can become even more than without exploitation. Equilibrium harvesting with the optimal harvest rate that maximizes yield leads to a population size decrease. The effect of hydra is at lower values of the catch quota than the optimal harvest rate. At the same time, the consequence of the hydra effect may be a higher abundance of the age-sex group under optimal exploitation compared to the level observed in the absence of harvesting.

The traditional approach to computing optimal game strategies of firms on oligopolistic markets and of indicators of such markets consists in studying linear dynamical games with quadratic criteria and solving generalized matrix Riccati equations.

The other approach proposed by the author is based on methods of operational calculus (in particular, Z-transform). This approach makes it possible to achieve economic meaningful decisions under wider field of parameter values. It characterizes by simplicity of computations and by necessary for economic analysis visibility. One of its advantages is that in many cases important for economic practice, it, in contrast to the traditional approach, provides the ability to make calculations using widespread spreadsheets, which allows to study the prospects for the development of oligopolistic markets to a wide range of professionals and consumers.

The article deals with the practical aspects of determining the optimal Nash–Cournot strategies of participants in oligopolistic markets on the basis of operational calculus, in particular the technique of computing the optimal Nash–Cournot strategies in Excel. As an illustration of the opportinities of the proposed methods of calculation, examples close to the practical problems of forecasting indicators of the markets of high-tech products are studied.

The results of calculations obtained by the author for numerous examples and real economic systems, both using the obtained relations on the basis of spreadsheets and using extended Riccati equations, are very close. In most of the considered practical problems, the deviation of the indicators calculated in accordance with the two approaches, as a rule, does not exceed 1.5–2%. The highest value of relative deviations (up to 3–5%) is observed at the beginning of the forecasting period. In typical cases, the period of relatively noticeable deviations is 3–5 moments of time. After the transition period, there is almost complete agreement of the values of the required indicators using both approaches.

Optimal management of fuel supply system boils down to choosing an energy development strategy which provides consumers with the most efficient and reliable fuel and energy supply. As a part of the program on switching the heat supply distributed management system of the Udmurt Republic to renewable energy sources, an “Information-analytical system of regional alternative fuel supply management” was developed. The paper presents the mathematical model of optimal management of fuel supply logistic system consisting of three interconnected levels: raw material accumulation points, fuel preparation points and fuel consumption points, which are heat sources. In order to increase effective the performance of regional fuel supply system a modification of information-analytical system and extension of its set of functions using the methods of quick responding when emergency occurs are required. Emergencies which occur on any one of these levels demand the management of the whole system to reconfigure. The paper demonstrates models and algorithms of optimal management in case of emergency involving break down of such production links of logistic system as raw material accumulation points and fuel preparation points. In mathematical models, the target criterion is minimization of costs associated with the functioning of logistic system in case of emergency. The implementation of the developed algorithms is based on the usage of genetic optimization algorithms, which made it possible to obtain a more accurate solution in less time. The developed models and algorithms are integrated into the information-analytical system that enables to provide effective management of alternative fuel supply of the Udmurt Republic in case of emergency.

Control theory methods for creating market structures are discussed for two cases: when market participants are pursuing aims 1) of maximal growth and 2) of maximum economic efficiency of their firms. For the first case method based on variable structure systems principles is developed. For the second case dynamic game approach is proposed based on computation of Nash–Cournot and Stackelberg strategies with the help of Z-transform.

The purpose of the study is to simulate the dynamics of traffic flows on city road networks as well as to systematize the current state of affairs in this area. The introduction states that the development of intelligent transportation systems as an integral part of modern transportation technologies is coming to the fore. The core of these systems contain adequate mathematical models that allow to simulate traffic as close to reality as possible. The necessity of using supercomputers due to the large amount of calculations is also noted, therefore, the creation of special parallel algorithms is needed. The beginning of the article is devoted to the up-to-date classification of traffic flow models and characterization of each class, including their distinctive features and relevant examples with links. Further, the main focus of the article is shifted towards the development of macroscopic and microscopic models, created by the authors, and determination of the place of these models in the aforementioned classification. The macroscopic model is based on the continuum approach and uses the ideology of quasi-gasdynamic systems of equations. Its advantages are indicated in comparison with existing models of this class. The model is presented both in one-dimensional and two-dimensional versions. The both versions feature the ability to study multi-lane traffic. In the two-dimensional version it is made possible by introduction of the concept of “lateral” velocity, i. e., the speed of changing lanes. The latter version allows for carrying out calculations in the computational domain which corresponds to the actual geometry of the road. The section also presents the test results of modeling vehicle dynamics on a road fragment with the local widening and on a road fragment with traffic lights, including several variants of traffic light regimes. In the first case, the calculations allow to draw interesting conclusions about the impact of a road widening on a road capacity as a whole, and in the second case — to select the optimal regime configuration to obtain the “green wave” effect. The microscopic model is based on the cellular automata theory and the single-lane Nagel – Schreckenberg model and is generalized for the multi-lane case by the authors of the article. The model implements various behavioral strategies of drivers. Test computations for the real transport network section in Moscow city center are presented. To achieve an adequate representation of vehicles moving through the network according to road traffic regulations the authors implemented special algorithms adapted for parallel computing. Test calculations were performed on the K-100 supercomputer installed in the Centre of Collective Usage of KIAM RAS.