All issues

The article deals with the problem of a distributed system operation reliability. The system core is an open integration platform that provides interaction of varied software for modeling gas transportation. Some of them provide an access through thin clients on the cloud technology “software as a service”. Mathematical models of operation, transmission and computing are to ensure the operation of an automated dispatching system for oil and gas transportation. The paper presents a system solution based on the theory of Markov random processes and considers the stable operation stage. The stationary operation mode of the Markov chain with continuous time and discrete states is described by a system of Chapman–Kolmogorov equations with respect to the average numbers (mathematical expectations) of the objects in certain states. The objects of research are both system elements that are present in a large number – thin clients and computing modules, and individual ones – a server, a network manager (message broker). Together, they are interacting Markov random processes. The interaction is determined by the fact that the transition probabilities in one group of elements depend on the average numbers of other elements groups.

The authors propose a multi-criteria dispersion model of risk assessment for such systems (both in the broad and narrow sense, in accordance with the IEC standard). The risk is the standard deviation of estimated object parameter from its average value. The dispersion risk model makes possible to define optimality criteria and whole system functioning risks. In particular, for a thin client, the following is calculated: the loss profit risk, the total risk of losses due to non-productive element states, and the total risk of all system states losses.

Finally the paper proposes compromise schemes for solving the multi-criteria problem of choosing the optimal operation strategy based on the selected set of compromise criteria.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

The paper proposes an approach to simulation of a sport game, consisting of a discrete set of separate competitions. According to this approach, such a competition is considered as a random processes, generally — a non-Markov’s one. At first we treat the flow of the game as a Markov’s process, obtaining recursive relationship between the probabilities of achieving certain states of score in a tennis match, as well as secondary indicators of the game, such as expectation and variance of the number of serves to finish the game. Then we use a simulation system, modeling the match, to allow an arbitrary change of the probabilities of the outcomes in the competitions that compose the match. We, for instance, allow the probabilities to depend on the results of previous competitions. Therefore, this paper deals with a modification of the model, previously proposed by the authors for sports games with continuous time.

The proposed approach allows to evaluate not only the probability of the final outcome of the match, but also the probabilities of reaching each of the possible intermediate results, as well as secondary indicators of the game, such as the number of separate competitions it takes to finish the match. The paper includes a detailed description of the construction of a simulation system for a game of a tennis match. Then we consider simulating a set and the whole tennis match by analogy. We show some statements concerning fairness of tennis serving rules, understood as independence of the outcome of a competition on the right to serve first. We perform simulation of a cancelled ATP series match, obtaining its most probable intermediate and final outcomes for three different possible variants of the course of the match.

The main result of this paper is the developed method of simulation of the match, applicable not only to tennis, but also to other types of sports games with discrete time.

The paper gives an overview of methods for estimating the parameters of random point fields with local interaction between points. It is shown that the conventional method of the maximum pseudo-likelihood is a special case of the family of estimation methods based on the use of the auxiliary Markov process, invariant measure of which is the Gibbs point field with parameters to be estimated. A generalization of this method, resulting in estimating equation that can not be obtained by the the universal Takacs–Fiksel method, is proposed. It is shown by computer simulations that the new method enables to obtain estimates which have better quality than those by a widely used method of the maximum pseudolikelihood.

In the present article we explore the process of changing the level of approval of a political leader under the influence of the processes taking place in online platforms (social networks, forums, etc.). The driver of these changes is the interaction of users, through which they can exchange opinions with each other and formulate their position in relation to the political leader. In addition to interpersonal interaction, we will consider such factors as the information impact, expressed in the creation of an information flow with a given power and polarity (positive or negative, in the context of influencing the image of a political leader), as well as the presence of a group of agents (opinion leaders), supporting the leader, or, conversely, negatively affecting its representation in the media space.

The mathematical basis of the presented research is the Kirman model, which has its roots in biology and initially found its application in economics. Within the framework of this model it is considered that each user is in one of the two possible states, and a Markov jump process describing transitions between these states is given. For the problem under consideration, these states are 0 or 1, depending on whether a particular agent is a supporter of a political leader or not. For further research, we find its diffusional approximation, known as the Jacoby process. With the help of spectral decomposition for the infinitesimal operator of this process we have an opportunity to find an analytical representation for the transition probability density.

Analyzing the probabilities obtained in this way, we can assess the influence of individual factors of the model: the power and direction of the information flow, available to online users and relevant to the tasks of rating formation, as well as the number of supporters or opponents of the politician. Next, using the found eigenfunctions and eigenvalues, we derive expressions for the evaluation of conditional mathematical expectations of a politician’s rating, which can serve as a basis for building forecasts that are important for the formation of a strategy of representing a political leader in the online environment.

In many social groups, for example, in technical committees for standardization, at the international, regional and national levels, in European communities, managers of ecovillages, social movements (occupy), international organizations, decision-making is based on the consensus of the group members. Instead of voting, where the majority wins over the minority, consensus allows for a solution that each member of the group supports, or at least considers acceptable. This approach ensures that all group members’ opinions, ideas and needs are taken into account. At the same time, it is noted that reaching consensus takes a long time, since it is necessary to ensure agreement within the group, regardless of its size. It was shown that in some situations the number of iterations (agreements, negotiations) is very significant. Moreover, in the decision-making process, there is always a risk of blocking the decision by the minority in the group, which not only delays the decisionmaking time, but makes it impossible. Typically, such a minority is one or two odious people in the group. At the same time, such a member of the group tries to dominate in the discussion, always remaining in his opinion, ignoring the position of other colleagues. This leads to a delay in the decision-making process, on the one hand, and a deterioration in the quality of consensus, on the other, since only the opinion of the dominant member of the group has to be taken into account. To overcome the crisis in this situation, it was proposed to make a decision on the principle of «consensus minus one» or «consensus minus two», that is, do not take into account the opinion of one or two odious members of the group.

The article, based on modeling consensus using the model of regular Markov chains, examines the question of how much the decision-making time according to the «consensus minus one» rule is reduced, when the position of the dominant member of the group is not taken into account.

The general conclusion that follows from the simulation results is that the rule of thumb for making decisions on the principle of «consensus minus one» has a corresponding mathematical justification. The simulation results showed that the application of the «consensus minus one» rule can reduce the time to reach consensus in the group by 76–95%, which is important for practice.

The average number of agreements hyperbolically depends on the average authoritarianism of the group members (excluding the authoritarian one), which means the possibility of delaying the agreement process at high values of the authoritarianism of the group members.

The theoretical study of consensus makes it possible to analyze the various situations that social groups that make decisions in this way have to face in real life, abstracting from the specific characteristics of the groups. It is relevant for practice to study the dynamics of a social group consisting of loyal experts who, in the process of seeking consensus, yield to each other. In this case, psychological “traps” such as false consensus or groupthink are possible, which can sometimes lead to managerial decisions with dire consequences.

The article builds a mathematical consensus model for a group of loyal experts based on modeling using regular Markov chains. Analysis of the model showed that with an increase in the loyalty (decrease in authoritarianism) of group members, the time to reach consensus increases exponentially (the number of agreements increases), which is apparently due to the lack of desire among experts to take part of the responsibility for the decision being made. An increase in the size of such a group leads (ceteris paribus):

– to reduce the number of approvals to consensus in the conditions of striving for absolute loyalty of members, i. e. each additional loyal member adds less and less “strength” to the group;

– to a logarithmic increase in the number of approvals in the context of an increase in the average authoritarianism of members. It is shown that in a small group (two people), the time for reaching consensus can increase by more than 10 times compared to a group of 5 or more members), in the group there is a transfer of responsibility for making decisions.

It is proved that in the case of a group of two absolutely loyal members, consensus is unattainable.

A reasonable conclusion is made that consensus in a group of loyal experts is a special (special) case of consensus, since the dependence of the time until consensus is reached on the authoritarianism of experts and their number in the group is described by different curves than in the case of a regular group of experts.

Often decisions in social groups are made by consensus. This applies, for example, to the examination in the technical committee for standardization (TC) before the approval of the national standard by Rosstandart. The standard is approved if and only if the secured consensus in the TC. The same approach to standards development was adopted in almost all countries and at the regional and international level. Previously published works of authors dedicated to the construction of a mathematical model of time to reach consensus in technical committees for standardization in terms of variation in the number of TC members and their level of authoritarianism. The present study is a continuation of these works for the case of the formation of coalitions that are often formed during the consideration of the draft standard to the TC. In the article the mathematical model is constructed to ensure consensus on the work of technical standardization committees in terms of coalitions. In the framework of the model it is shown that in the presence of coalitions consensus is not achievable. However, the coalition, as a rule, are overcome during the negotiation process, otherwise the number of the adopted standards would be extremely small. This paper analyzes the factors that influence the bridging coalitions: the value of the assignment and an index of the effect of the coalition. On the basis of statistical modelling of regular Markov chains is investigated their effects on the time to ensure consensus in the technical Committee. It is proved that the time to reach consensus significantly depends on the value of unilateral concessions coalition and weakly depends on the size of coalitions. Built regression model of dependence of the average number of approvals from the value of the assignment. It was revealed that even a small concession leads to the onset of consensus, increasing the size of the assignment results (with other factors being equal) to a sharp decline in time before the consensus. It is shown that the assignment of a larger coalition against small coalitions takes on average more time before consensus. The result has practical value for all organizational structures, where the emergence of coalitions entails the inability of decision-making in the framework of consensus and requires the consideration of various methods for reaching a consensus decision.