All issues

The paper is devoted to convex-concave saddle point problems where the objective is a sum of a large number of functions. Such problems attract considerable attention of the mathematical community due to the variety of applications in machine learning, including adversarial learning, adversarial attacks and robust reinforcement learning, to name a few. The individual functions in the sum usually represent losses related to examples from a data set. Additionally, the formulation admits a possibly nonsmooth composite term. Such terms often reflect regularization in machine learning problems. We assume that the dimension of one of the variable groups is relatively small (about a hundred or less), and the other one is large. This case arises, for example, when one considers the dual formulation for a minimization problem with a moderate number of constraints. The proposed approach is based on using Vaidya’s cutting plane method to minimize with respect to the outer block of variables. This optimization algorithm is especially effective when the dimension of the problem is not very large. An inexact oracle for Vaidya’s method is calculated via an approximate solution of the inner maximization problem, which is solved by the accelerated variance reduced algorithm Katyusha. Thus, we leverage the structure of the problem to achieve fast convergence. Separate complexity bounds for gradients of different components with respect to different variables are obtained in the study. The proposed approach is imposing very mild assumptions about the objective. In particular, neither strong convexity nor smoothness is required with respect to the low-dimensional variable group. The number of steps of the proposed algorithm as well as the arithmetic complexity of each step explicitly depend on the dimensionality of the outer variable, hence the assumption that it is relatively small.

This paper presents the results of experimental validation of some structural issues concerning the practical use of methods to overcome catastrophic forgetting of neural networks. A comparison of current effective methods like EWC (Elastic Weight Consolidation) and WVA (Weight Velocity Attenuation) is made and their advantages and disadvantages are considered. It is shown that EWC is better for tasks where full retention of learned skills is required on all the tasks in the training queue, while WVA is more suitable for sequential tasks with very limited computational resources, or when reuse of representations and acceleration of learning from task to task is required rather than exact retention of the skills. The attenuation of the WVA method must be applied to the optimization step, i. e. to the increments of neural network weights, rather than to the loss function gradient itself, and this is true for any gradient optimization method except the simplest stochastic gradient descent (SGD). The choice of the optimal weights attenuation function between the hyperbolic function and the exponent is considered. It is shown that hyperbolic attenuation is preferable because, despite comparable quality at optimal values of the hyperparameter of the WVA method, it is more robust to hyperparameter deviations from the optimal value (this hyperparameter in the WVA method provides a balance between preservation of old skills and learning a new skill). Empirical observations are presented that support the hypothesis that the optimal value of this hyperparameter does not depend on the number of tasks in the sequential learning queue. And, consequently, this hyperparameter can be picked up on a small number of tasks and used on longer sequences.

A model of combustion of premixed mixture of gases with one global chemical reaction is considered, the model includes equations of the second order for temperature of mixture and concentrations of fuel and oxidizer, and the right-hand sides of these equations contain the reaction rate function. This function depends on five unknown parameters of the global reaction and serves as approximation to multistep reaction mechanism. The model is reduced, after replacement of variables, to one equation of the second order for temperature of mixture that transforms to a first-order equation for temperature derivative depending on temperature that contains a parameter of flame propagation velocity. Thus, for computing the parameter of burning velocity, one has to solve Dirichlet problem for first-order equation, and after that a model dependence of burning velocity on mixture equivalence ratio at specified reaction rate parameters will be obtained. Given the experimental data of dependence of burning velocity on mixture equivalence ratio, the problem of optimal selection of reaction rate parameters is stated, based on minimization of the mean square deviation of model values of burning velocity on experimental ones. The aim of our study is analysis of uniqueness of this problem solution. To this end, we apply computational experiment during which the problem of global search of optima is solved using multistart of gradient descent. The computational experiment clarifies that the inverse problem in this statement is underdetermined, and every time, when running gradient descent from a selected starting point, it converges to a new limit point. The structure of the set of limit points in the five-dimensional space is analyzed, and it is shown that this set can be described with three linear equations. Therefore, it might be incorrect to tabulate all five parameters of reaction rate based on just one match criterion between model and experimental data of flame propagation velocity. The conclusion of our study is that in order to tabulate reaction rate parameters correctly, it is necessary to specify the values of two of them, based on additional optimality criteria.

The paper reviews the existing approaches to calculating the destruction of solids. The main attention is paid to algorithms using a unified approach to the calculation of deformation both for nondestructive and for the destroyed states of the material. The thermodynamic derivation of the unified rheological relationships taking into account the elastic, viscous and plastic properties of materials and describing the loss of the deformation resistance ability with the accumulation of microdamages is presented. It is shown that the mathematical model under consideration provides a continuous dependence of the solution on input parameters (parameters of the material medium, initial and boundary conditions, discretization parameters) with softening of the material.

Explicit and implicit non-matrix algorithms for calculating the evolution of deformation and fracture development are presented. Non-explicit schemes are implemented using iterations of the conjugate gradient method, with the calculation of each iteration exactly coinciding with the calculation of the time step for two-layer explicit schemes. So, the solution algorithms are very simple.

The results of solving typical problems of destruction of solid deformable bodies for slow (quasistatic) and fast (dynamic) deformation processes are presented. Based on the experience of calculations, recommendations are given for modeling the processes of destruction and ensuring the reliability of numerical solutions.

The purpose of this work is to develop a universal computer program (solver) which solves kinetic Boltzmann equation for simulations of rarefied gas flows in complexly shaped devices. The structure of the solver is described in details. Its efficiency is demonstrated on an example of calculations of a modern many tubes Knudsen pump. The kinetic Boltzmann equation is solved by finite-difference method on discrete grid in spatial and velocity spaces. The differential advection operator is approximated by finite difference method. The calculation of the collision integral is based on the conservative projection method.

In the developed computational program the unstructured spatial mesh is generated using GMSH and may include prisms, tetrahedrons, hexahedrons and pyramids. The mesh is denser in areas of flow with large gradients of gas parameters. A three-dimensional velocity grid consists of cubic cells of equal volume.

A huge amount of calculations requires effective parallelization of the algorithm which is implemented in the program with the use of Message Passing Interface (MPI) technology. An information transfer from one node to another is implemented as a kind of boundary condition. As a result, every MPI node contains the information about only its part of the grid.

The main result of the work is presented in the graph of pressure difference in 2 reservoirs connected by a multitube Knudsen pump from Knudsen number. This characteristic of the Knudsen pump obtained by numerical methods shows the quality of the pump. Distributions of pressure, temperature and gas concentration in a steady state inside the pump and the reservoirs are presented as well.

The correctness of the solver is checked using two special test solutions of more simple boundary problems — test with temperature distribution between 2 planes with different temperatures and test with conservation of total gas mass.

The correctness of the obtained data for multitube Knudsen pump is checked using denser spatial and velocity grids, using more collisions in collision integral per time step.

Inverse geophysical problems are difficult to solve due to their mathematically incorrect formulation and large computational complexity. Geophysical exploration in frontier areas is even more complicated due to the lack of reliable geological information. In this case, inversion methods that allow interpretation of several types of geophysical data together are recognized to be of major importance. This paper is dedicated to one of such inversion methods, which is based on minimization of the determinant of the Gram matrix for a set of model vectors. Within the framework of this approach, we minimize a nonlinear functional, which consists of squared norms of data residual of different types, the sum of stabilizing functionals and a term that measures the structural similarity between different model vectors. We apply this approach to seismic and electromagnetic synthetic data set. Specifically, we study joint inversion of acoustic pressure response together with controlled-source electrical field imposing structural constraints on resulting electrical conductivity and P-wave velocity distributions.

We start off this note with the problem formulation and present the numerical method for inverse problem. We implemented the conjugate-gradient algorithm for non-linear optimization. The efficiency of our approach is demonstrated in numerical experiments, in which the true 3D electrical conductivity model was assumed to be known, but the velocity model was constructed during inversion of seismic data. The true velocity model was based on a simplified geology structure of a marine prospect. Synthetic seismic data was used as an input for our minimization algorithm. The resulting velocity model not only fit to the data but also has structural similarity with the given conductivity model. Our tests have shown that optimally chosen weight of the Gramian term may improve resolution of the final models considerably.

The work is devoted to the construction of efficient and applicable to real tasks first-order methods of convex optimization, that is, using only values of the target function and its derivatives. Construction uses OGMG, fast gradient method which is optimal by complexity, but requires to know the Lipschitz constant for gradient and the strong convexity constant to determine the number of steps and step length. This requirement makes practical usage very hard. An adaptive on the constant for strong convexity algorithm ACGM is proposed, based on restarts of the OGM-G with update of the strong convexity constant estimate, and an adaptive on the Lipschitz constant for gradient ALGM, in which the use of OGM-G restarts is supplemented by the selection of the Lipschitz constant with verification of the smoothness conditions used in the universal gradient descent method. This eliminates the disadvantages of the original method associated with the need to know these constants, which makes practical usage possible. Optimality of estimates for the complexity of the constructed algorithms is proved. To verify the results obtained, experiments on model functions and real tasks from machine learning are carried out.

The article is devoted to solving ill-posed problems of mathematical physics for elliptic and parabolic equations, such as the Cauchy problem for the Helmholtz equation and the retrospective Cauchy problem for the heat equation with constant coefficients. These problems are reduced to problems of convex optimization in Hilbert space. The gradients of the corresponding functionals are calculated approximately by solving two well-posed problems. A new method is proposed for solving the optimization problems under study, it is component-by-component descent in the basis of eigenfunctions of a self-adjoint operator associated with the problem. If it was possible to calculate the gradient exactly, this method would give an arbitrarily exact solution of the problem, depending on the number of considered elements of the basis. In real cases, the inaccuracy of calculations leads to a violation of monotonicity, which requires the use of restarts and limits the achievable quality. The paper presents the results of experiments confirming the effectiveness of the constructed method. It is determined that the new approach is superior to approaches based on the use of gradient optimization methods: it allows to achieve better quality of solution with significantly less computational resources. It is assumed that the constructed method can be generalized to other problems.

An algorithm is proposed to identify parameters of a 2D vortex structure used on information about the flow velocity at a finite (small) set of reference points. The approach is based on using a set of point vortices as a model system and minimizing a functional that compares the model and known sets of velocity vectors in the space of model parameters. For numerical implementation, the method of gradient descent with step size control, approximation of derivatives by finite differences, and the analytical expression of the velocity field induced by the point vortex model are used. An experimental analysis of the operation of the algorithm on test flows is carried out: one and a system of several point vortices, a Rankine vortex, and a Lamb dipole. According to the velocity fields of test flows, the velocity vectors utilized for identification were arranged in a randomly distributed set of reference points (from 3 to 200 pieces). Using the computations, it was determined that: the algorithm converges to the minimum from a wide range of initial approximations; the algorithm converges in all cases when the reference points are located in areas where the streamlines of the test and model systems are topologically equivalent; if the streamlines of the systems are not topologically equivalent, then the percentage of successful calculations decreases, but convergence can also take place; when the method converges, the coordinates of the vortices of the model system are close to the centers of the vortices of the test configurations, and in many cases, the values of their circulations also; con-vergence depends more on location than on the number of vectors used for identification. The results of the study allow us to recommend the proposed algorithm for identifying 2D vortex structures whose streamlines are topologically close to systems of point vortices.

In this paper we consider the controlled motion of a helical body with three blades in an ideal fluid, which is executed by rotating three internal rotors. We set the problem of selecting control actions, which ensure the motion of the body near the predetermined trajectory. To determine controls that guarantee motion near the given curve, we propose methods based on the application of hybrid genetic algorithms (genetic algorithms with real encoding and with additional learning of the leader of the population by a gradient method) and artificial neural networks. The correctness of the operation of the proposed numerical methods is estimated using previously obtained differential equations, which define the law of changing the control actions for the predetermined trajectory.

In the approach based on hybrid genetic algorithms, the initial problem of minimizing the integral functional reduces to minimizing the function of many variables. The given time interval is broken up into small elements, on each of which the control actions are approximated by Lagrangian polynomials of order 2 and 3. When appropriately adjusted, the hybrid genetic algorithms reproduce a solution close to exact. However, the cost of calculation of 1 second of the physical process is about 300 seconds of processor time.

To increase the speed of calculation of control actions, we propose an algorithm based on artificial neural networks. As the input signal the neural network takes the components of the required displacement vector. The node values of the Lagrangian polynomials which approximately describe the control actions return as output signals . The neural network is taught by the well-known back-propagation method. The learning sample is generated using the approach based on hybrid genetic algorithms. The calculation of 1 second of the physical process by means of the neural network requires about 0.004 seconds of processor time, that is, 6 orders faster than the hybrid genetic algorithm. The control calculated by means of the artificial neural network differs from exact control. However, in spite of this difference, it ensures that the predetermined trajectory is followed exactly.