All issues

The article deals with the problem of a distributed system operation reliability. The system core is an open integration platform that provides interaction of varied software for modeling gas transportation. Some of them provide an access through thin clients on the cloud technology “software as a service”. Mathematical models of operation, transmission and computing are to ensure the operation of an automated dispatching system for oil and gas transportation. The paper presents a system solution based on the theory of Markov random processes and considers the stable operation stage. The stationary operation mode of the Markov chain with continuous time and discrete states is described by a system of Chapman–Kolmogorov equations with respect to the average numbers (mathematical expectations) of the objects in certain states. The objects of research are both system elements that are present in a large number – thin clients and computing modules, and individual ones – a server, a network manager (message broker). Together, they are interacting Markov random processes. The interaction is determined by the fact that the transition probabilities in one group of elements depend on the average numbers of other elements groups.

The authors propose a multi-criteria dispersion model of risk assessment for such systems (both in the broad and narrow sense, in accordance with the IEC standard). The risk is the standard deviation of estimated object parameter from its average value. The dispersion risk model makes possible to define optimality criteria and whole system functioning risks. In particular, for a thin client, the following is calculated: the loss profit risk, the total risk of losses due to non-productive element states, and the total risk of all system states losses.

Finally the paper proposes compromise schemes for solving the multi-criteria problem of choosing the optimal operation strategy based on the selected set of compromise criteria.

The paper considers the process of flow formation in an outflow of a binary gas mixture through a thin slit into vacuum. An approach to modeling the flows of rarefied gas mixtures in the transient regime is proposed based on the direct solution of the Boltzmann kinetic equation, in which the conservative projection method is used to calculate the collision integrals. Calculation formulas are provided; the calculation procedure is described in detail in relation to the flow of a binary gas mixture. The Lennard–Jones potential is used as an interaction potential of molecules. A software modeling environment has been developed that makes it possible to study the flows of gas mixtures in a transitional regime on systems of cluster architecture. Due to the use of code parallelization technologies, an acceleration of calculations by 50–100 times was obtained. Numerical simulation of a two-dimensional outflow of a binary argon-neon gas mixture from a vessel into vacuum through a thin slit is carried out for various values of the Knudsen number. The graphs of the dependence of gas mixture components output flow on time in the process of establishing the flow are obtained. Non-stationary regions of strong separation of gas mixture components, in which the molecular densities ratio reaches 10 or more, were discovered. The discovered effect can have applications in the problem of gas mixtures separation.

This article reviews both historical achievements and modern results in the field of Markov Decision Process (MDP) and convex optimization. This review is the first attempt to cover the field of reinforcement learning in Russian in the context of convex optimization. The fundamental Bellman equation and the criteria of optimality of policy — strategies based on it, which make decisions based on the known state of the environment at the moment, are considered. The main iterative algorithms of policy optimization based on the solution of the Bellman equations are also considered. An important section of this article was the consideration of an alternative to the $Q$-learning approach — the method of direct maximization of the agent’s average reward for the chosen strategy from interaction with the environment. Thus, the solution of this convex optimization problem can be represented as a linear programming problem. The paper demonstrates how the convex optimization apparatus is used to solve the problem of Reinforcement Learning (RL). In particular, it is shown how the concept of strong duality allows us to naturally modify the formulation of the RL problem, showing the equivalence between maximizing the agent’s reward and finding his optimal strategy. The paper also discusses the complexity of MDP optimization with respect to the number of state–action–reward triples obtained as a result of interaction with the environment. The optimal limits of the MDP solution complexity are presented in the case of an ergodic process with an infinite horizon, as well as in the case of a non-stationary process with a finite horizon, which can be restarted several times in a row or immediately run in parallel in several threads. The review also reviews the latest results on reducing the gap between the lower and upper estimates of the complexity of MDP optimization with average remuneration (Averaged MDP, AMDP). In conclusion, the real-valued parametrization of agent policy and a class of gradient optimization methods through maximizing the $Q$-function of value are considered. In particular, a special class of MDPs with restrictions on the value of policy (Constrained Markov Decision Process, CMDP) is presented, for which a general direct-dual approach to optimization with strong duality is proposed.

The paper proposes a mathematical model for the therapy of glioma, taking into account the blood-brain barrier, radiotherapy and antibody therapy. The parameters were estimated from experimental data and the evaluation of the effect of parameter values on the effectiveness of treatment and the prognosis of the disease were obtained. The possible variants of sequential use of radiotherapy and the effect of antibodies have been explored. The combined use of radiotherapy with intravenous administration of $mab$ $Cx43$ leads to a potentiation of the therapeutic effect in glioma.

Radiotherapy must precede chemotherapy, as radio exposure reduces the barrier function of endothelial cells. Endothelial cells of the brain vessels fit tightly to each other. Between their walls are formed so-called tight contacts, whose role in the provision of BBB is that they prevent the penetration into the brain tissue of various undesirable substances from the bloodstream. Dense contacts between endothelial cells block the intercellular passive transport.

The mathematical model consists of a continuous part and a discrete one. Experimental data on the volume of glioma show the following interesting dynamics: after cessation of radio exposure, tumor growth does not resume immediately, but there is some time interval during which glioma does not grow. Glioma cells are divided into two groups. The first group is living cells that divide as fast as possible. The second group is cells affected by radiation. As a measure of the health of the blood-brain barrier system, the ratios of the number of BBB cells at the current moment to the number of cells at rest, that is, on average healthy state, are chosen.

The continuous part of the model includes a description of the division of both types of glioma cells, the recovery of BBB cells, and the dynamics of the drug. Reducing the number of well-functioning BBB cells facilitates the penetration of the drug to brain cells, that is, enhances the action of the drug. At the same time, the rate of division of glioma cells does not increase, since it is limited not by the deficiency of nutrients available to cells, but by the internal mechanisms of the cell. The discrete part of the mathematical model includes the operator of radio interaction, which is applied to the indicator of BBB and to glial cells.

Within the framework of the mathematical model of treatment of a cancer tumor (glioma), the problem of optimal control with phase constraints is solved. The patient’s condition is described by two variables: the volume of the tumor and the condition of the BBB. The phase constraints delineate a certain area in the space of these indicators, which we call the survival area. Our task is to find such treatment strategies that minimize the time of treatment, maximize the patient’s rest time, and at the same time allow state indicators not to exceed the permitted limits. Since the task of survival is to maximize the patient’s lifespan, it is precisely such treatment strategies that return the indicators to their original position (and we see periodic trajectories on the graphs). Periodic trajectories indicate that the deadly disease is translated into a chronic one.

The biological structure is considered as an open nonequilibrium system which properties can be described on the basis of kinetic equations. New problems with nonequilibrium boundary conditions are introduced. The nonequilibrium distribution tends gradually to an equilibrium state. The region of spatial inhomogeneity has a scale depending on the rate of mass transfer in the open system and the characteristic time of metabolism. In the proposed approximation, the internal energy of the motion of molecules is much less than the energy of translational motion. Or in other terms we can state that the kinetic energy of the average blood velocity is substantially higher than the energy of chaotic motion of the same particles. We state that the relaxation problem models a living system. The flow of entropy to the system decreases in downstream, this corresponds to Shrödinger’s general ideas that the living system “feeds on” negentropy. We introduce a quantity that determines the complexity of the biosystem, more precisely, this is the difference between the nonequilibrium kinetic entropy and the equilibrium entropy at each spatial point integrated over the entire spatial region. Solutions to the problems of spatial relaxation allow us to estimate the size of biosystems as regions of nonequilibrium. The results are compared with empirical data, in particular, for mammals we conclude that the larger the size of animals, the smaller the specific energy of metabolism. This feature is reproduced in our model since the span of the nonequilibrium region is larger in the system where the reaction rate is shorter, or in terms of the kinetic approach, the longer the relaxation time of the interaction between the molecules. The approach is also used for estimation of a part of a living system, namely a green leaf. The problems of aging as degradation of an open nonequilibrium system are considered. The analogy is related to the structure, namely, for a closed system, the equilibrium of the structure is attained for the same molecules while in the open system, a transition occurs to the equilibrium of different particles, which change due to metabolism. Two essentially different time scales are distinguished, the ratio of which is approximately constant for various animal species. Under the assumption of the existence of these two time scales the kinetic equation splits in two equations, describing the metabolic (stationary) and “degradative” (nonstationary) parts of the process.

The article reports the findings of an investigation into pollution migration processes at the municipal solid waste (MSW) landfill located in the water protection zone of Lake Seliger (Tver Region). The distribution of pollutants is investigated and migration parameters are determined in field and laboratory conditions at the landfill site. A mathematical model describing physical and chemical processes of substance migration in soil strata is constructed. Pollutant migration is found to be due to a variety of factors. The major ones, having a significant impact on the migration of MSW ingredients and taken into account mathematically, include convective transport, diffusion and sorption processes. A modified mathematical model differs from its conventional counterparts by considering a number of parameters reflecting the decrease in the concentration of ammonium and nitrate nitrogen ions in ground water (transpiration by plant roots, dilution with infiltration waters, etc.). An analytical solution to assess the pollutant spread from the landfill is presented. The mathematical model provides a set of simulation models helping to obtain a computational solution of specific problems, vertical and horizontal migration of substances in the underground flow. Numerical experiments, analytical solutions, as well as field and laboratory data was studied the dynamics of pollutant distribution in the object under study up to the lake. A long-term forecast for the spread of landfill pollution is made. Simulation experiments showed that some zones of clean groundwater interact with those of contaminated groundwater during the pollution migration from the landfill, each characterized by a different pollutant content. The data of a computational experiments and analytical calculations are consistent with the findings of field and laboratory investigations of the object and give grounds to recommend the proposed models for predicting pollution migration from a landfill. The analysis of the pollution migration simulation allows to substantiate the numerical estimates of the increase in $NH_4^+$ and $NO_3^-$ ion concentration with the landfill operation time. It is found that, after 100 years following the landfill opening, toxic filtrate components will fill the entire pore space from the landfill to the lake resulting in a significant deterioration of the ecosystem of Lake Seliger.

The paper analyzes the methods of studying the process of interaction of soil environments with the tillage tools of soil processing machines. The mathematical methods of numerical modeling are considered in detail, which make it possible to overcome the disadvantages of analytical and empirical approaches. A classification and overview of the possibilities the continuous (FEM — finite element method, CFD — computational fluid dynamics) and discrete (DEM — discrete element method, SPH — hydrodynamics of smoothed particles) numerical methods is presented. Based on the discrete element method, a mathematical model has been developed that represents the soil in the form of a set of interacting small spherical elements. The working surfaces of the tillage tool are presented in the framework of the finite element approximation in the form of a combination of many elementary triangles. The model calculates the movement of soil elements under the action of contact forces of soil elements with each other and with the working surfaces of the tillage tool (elastic forces, dry and viscous friction forces). This makes it possible to assess the influence of the geometric parameters of the tillage tools, technological parameters of the process and soil parameters on the geometric indicators of soil displacement, indicators of the self-installation of tools, power loads, quality indicators of loosening and spatial distribution of indicators. A total of 22 indicators were investigated (or the distribution of the indicator in space). This makes it possible to reproduce changes in the state of the system of elements of the soil (soil cultivation process) and determine the total mechanical effect of the elements on the moving tillage tools of the implement. A demonstration of the capabilities of the mathematical model is given by the example of a study of soil cultivation with a disk cultivator battery. In the computer experiment, a virtual soil channel of 5×1.4 m in size and a 3D model of a disk cultivator battery were used. The radius of the soil particles was taken to be 18 mm, the speed of the tillage tool was 1 m/s, the total simulation time was 5 s. The processing depth was 10 cm at angles of attack of 10, 15, 20, 25 and 30°. The verification of the reliability of the simulation results was carried out on a laboratory stand for volumetric dynamometry by examining a full-scale sample, made in full accordance with the investigated 3D-model. The control was carried out according to three components of the traction resistance vector: $F_x$, $F_y$ and $F_z$. Comparison of the data obtained experimentally with the simulation data showed that the discrepancy is not more than 22.2%, while in all cases the maximum discrepancy was observed at angles of attack of the disk battery of 30°. Good consistency of data on three key power parameters confirms the reliability of the whole complex of studied indicators.

The repressilator is the first genetic regulatory network in synthetic biology, which was artificially constructed in 2000. It is a closed network of three genetic elements $lacI$, $\lambda cI$ and $tetR$, which have a natural origin, but are not found in nature in such a combination. The promoter of each of the three genes controls the next cistron via the negative feedback, suppressing the expression of the neighboring gene. In our previous paper [Bratsun et al., 2018], we proposed a mathematical model of a delayed repressillator and studied its properties within the framework of a deterministic description. We assume that delay can be both natural, i.e. arises during the transcription / translation of genes due to the multistage nature of these processes, and artificial, i.e. specially to be introduced into the work of the regulatory network using gene engineering technologies. In this work, we apply the stochastic description of dynamic processes in a delayed repressilator, which is an important addition to deterministic analysis due to the small number of molecules involved in gene regulation. The stochastic study is carried out numerically using the Gillespie algorithm, which is modified for time delay systems. We present the description of the algorithm, its software implementation, and the results of benchmark simulations for a onegene delayed autorepressor. When studying the behavior of a repressilator, we show that a stochastic description in a number of cases gives new information about the behavior of a system, which does not reduce to deterministic dynamics even when averaged over a large number of realizations. We show that in the subcritical range of parameters, where deterministic analysis predicts the absolute stability of the system, quasi-regular oscillations may be excited due to the nonlinear interaction of noise and delay. Earlier, we have discovered within the framework of the deterministic description, that there exists a long-lived transient regime, which is represented in the phase space by a slow manifold. This mode reflects the process of long-term synchronization of protein pulsations in the work of the repressilator genes. In this work, we show that the transition to the cooperative mode of gene operation occurs a two order of magnitude faster, when the effect of the intrinsic noise is taken into account. We have obtained the probability distribution of moment when the phase trajectory leaves the slow manifold and have determined the most probable time for such a transition. The influence of the intrinsic noise of chemical reactions on the dynamic properties of the repressilator is discussed.

Technical carbon (soot) is a product obtained by thermal decomposition (pyrolysis) of hydrocarbons (usually oil) in a stream of heat carrier gas. Technical carbon is widely used as a reinforcing component in the production of rubber and plastic masses. Tire production uses 70% of all carbon produced. In furnace carbon production, the liquid hydrocarbon feedstock is injected into the natural gas combustion product stream through nozzles. The raw material is atomized and vaporized with further pyrolysis. It is important for the raw material to be completely evaporated before the pyrolysis process starts, otherwise coke, that contaminates the product, will be produced. It is impossible to operate without mathematical modeling of the process itself in order to improve the carbon production technology, in particular, to provide the complete evaporation of the raw material prior to the pyrolysis process. Mathematical modelling is the most important way to obtain the most complete and detailed information about the peculiarities of reactor operation.

A three-dimensional mathematical model and calculation method for raw material atomization and evaporation in the thermal gas flow are being developed in the FlowVision software package PC. Water is selected as a raw material to work out the modeling technique. The working substances in the reactor chamber are the combustion products of natural gas. The motion of raw material droplets and evaporation in the gas stream are modeled in the framework of the Eulerian approach of interaction between dispersed and continuous media. The simulation results of raw materials atomization and evaporation in a real reactor for technical carbon production are presented. Numerical method allows to determine an important atomization characteristic: average Sauter diameter. That parameter could be defined from distribution of droplets of raw material at each time of spray forming.

The functioning of DNA regulatory regions rely primarily on their physicochemical and structural properties but not on nucleotide sequences, i.e. ‘genetic text’. The formers are responsible for coding of DNA-protein interactions that govern various regulatory events. One of the characteristics is SIDD (Stress-Induced Duplex Destabilization) that quantify DNA duplex region propensity to melt under the imposed superhelical stress. The duplex property has been shown to participate in activity of various regulatory regions. Here we employ the SIDD model to calculate melting probability profiles for T7 bacteriophage promoter sequences. The genome is characterized by small size (approximately 40 thousand nucleotides) and temporal organization of expression: at the first stage of infection early T7 DNA region is transcribed by the host cell RNA polymerase, later on in life cycle phage-specific RNA polymerase performs transcription of class II and class III genes regions. Differential recognition of a particular group of promoters by the enzyme cannot be solely explained by their nucleotide sequences, because of, among other reasons, it is fairly similar among most the promoters. At the same time SIDD profiles obtained vary significantly and are clearly separated into groups corresponding to functional promoter classes of T7 DNA. For example, early promoters are affected by the same maximally destabilized DNA duplex region located at the varying region of a particular promoter. class II promoters lack substantially destabilized regions close to transcription start sites. Class III promoters, in contrast, demonstrate characteristic melting probability maxima located in the near-downstream region in all cases. Therefore, the apparent differences among the promoter groups with exceptional textual similarity (class II and class III differ by only few singular substitutions) were established. This confirms the major impact of DNA primary structure on the duplex parameter as well as a need for a broad genetic context consideration. The differences in melting probability profiles obtained using SIDD model alongside with other DNA physicochemical properties appears to be involved in differential promoter recognition by RNA polymerases.