All issues

The article considers minimization of the expectation of convex function. Problems of this type often arise in machine learning and a variety of other applications. In practice, stochastic gradient descent (SGD) and similar procedures are usually used to solve such problems. We propose to use the ellipsoid method with mini-batching, which converges linearly and can be more efficient than SGD for a class of problems. This is verified by our experiments, which are publicly available. The algorithm does not require neither smoothness nor strong convexity of the objective to achieve linear convergence. Thus, its complexity does not depend on the conditional number of the problem. We prove that the method arrives at an approximate solution with given probability when using mini-batches of size proportional to the desired accuracy to the power −2. This enables efficient parallel execution of the algorithm, whereas possibilities for batch parallelization of SGD are rather limited. Despite fast convergence, ellipsoid method can result in a greater total number of calls to oracle than SGD, which works decently with small batches. Complexity is quadratic in dimension of the problem, hence the method is suitable for relatively small dimensionalities.

This article is dedicated to use of S-spline theory for solving equations in partial derivatives. For example, we consider solution of the Poisson equation. S-spline — is a piecewise-polynomial function. Its coefficients are defined by two states. The first part of coefficients are defined by smoothness of the spline. The second coefficients are determined by least-squares method. According to order of considered polynomial and number of conditions of first and second type we get S-splines with different properties. At this moment we have investigated order 3 S-splines of class C¹ and order 5 S-splines of class C² (they meet conditions of smoothness of order 1 and 2 respectively). We will consider how the order 3 S-splines of class C¹ can be applied for solving equation of Poisson on circle and other areas.

We obtain asymptotic equalities for upper bounds of the deviations of the right-angled de la Vallee Poussin sums taken over classes of periodical functions of two variables of high smoothness. These equalities guarantee the solvability of the Kolmogorov–Nikol’skii problem for the right-angled de la Vallee Poussin sums on the specified classes of functions.

We obtain asymptotic equalities for upper bounds of the deviations of the right-angled de la Vallee Poussin sums taken over classes of periodical functions of many variables of high smoothness. These equalities guarantee the solvability of the Kolmogorov–Nikol’skii problem for the right-angled de la Vallee Poussin sums on the specified classes of functions.

WENO schemes (weighted, essentially non oscillating) are currently having a wide range of applications as approximate high order schemes for discontinuous solutions of partial differential equations. These schemes are used for direct numerical simulation (DNS) and large eddy simmulation in the gas dynamic problems, problems for DNS in MHD and even neutron kinetics. This work is dedicated to clarify some characteristics of WENO schemes and numerical simulation of specific tasks. Results of the simulations can be used to clarify the field of application of these schemes. The first part of the work contained proofs of the approximation properties, stability and convergence of WENO5, WENO7, WENO9, WENO11 and WENO13 schemes. In the second part of the work the modified wave number analysis is conducted that allows to conclude the dispersion and dissipative properties of schemes. Further, a numerical simulation of a number of specific problems for hyperbolic equations is conducted, namely for advection equations (one-dimensional and two-dimensional), Hopf equation, Burgers equation (with low dissipation) and equations of non viscous gas dynamics (onedimensional and two-dimensional). For each problem that is implying a smooth solution, the practical calculation of the order of approximation via Runge method is performed. The influence of a time step on nonlinear properties of the schemes is analyzed experimentally in all problems and cross checked with the first part of the paper. In particular, the advection equations of a discontinuous function and Hopf equations show that the failure of the recommendations from the first part of the paper leads first to an increase in total variation of the solution and then the approximation is decreased by the non-linear dissipative mechanics of the schemes. Dissipation of randomly distributed initial conditions in a periodic domain for one-dimensional Burgers equation is conducted and a comparison with the spectral method is performed. It is concluded that the WENO7–WENO13 schemes are suitable for direct numerical simulation of turbulence. At the end we demonstrate the possibility of the schemes to be used in solution of initial-boundary value problems for equations of non viscous gas dynamics: Rayleigh–Taylor instability and the reflection of the shock wave from a wedge with the formation a complex configuration of shock waves and discontinuities.

In this paper, we introduce a new version of Gauss–Newton method for solving a system of nonlinear equations based on ideas of the residual upper bound for a system of nonlinear equations and a quadratic regularization term. The introduced Gauss–Newton method in practice virtually forms the whole parameterized family of the methods solving systems of nonlinear equations and regression problems. The developed family of Gauss–Newton methods completely consists of iterative methods with generalization for cases of non-euclidean normed spaces, including special forms of Levenberg–Marquardt algorithms. The developed methods use the local model based on a parameterized proximal mapping allowing us to use an inexact oracle of «black–box» form with restrictions for the computational precision and computational complexity. We perform an efficiency analysis including global and local convergence for the developed family of methods with an arbitrary oracle in terms of iteration complexity, precision and complexity of both local model and oracle, problem dimensionality. We present global sublinear convergence rates for methods of the proposed family for solving a system of nonlinear equations, consisting of Lipschitz smooth functions. We prove local superlinear convergence under extra natural non-degeneracy assumptions for system of nonlinear functions. We prove both local and global linear convergence for a system of nonlinear equations under Polyak–Lojasiewicz condition for proposed Gauss– Newton methods. Besides theoretical justifications of methods we also consider practical implementation issues. In particular, for conducted experiments we present effective computational schemes for the exact oracle regarding to the dimensionality of a problem. The proposed family of methods unites several existing and frequent in practice Gauss–Newton method modifications, allowing us to construct a flexible and convenient method implementable using standard convex optimization and computational linear algebra techniques.

The paper is devoted to convex-concave saddle point problems where the objective is a sum of a large number of functions. Such problems attract considerable attention of the mathematical community due to the variety of applications in machine learning, including adversarial learning, adversarial attacks and robust reinforcement learning, to name a few. The individual functions in the sum usually represent losses related to examples from a data set. Additionally, the formulation admits a possibly nonsmooth composite term. Such terms often reflect regularization in machine learning problems. We assume that the dimension of one of the variable groups is relatively small (about a hundred or less), and the other one is large. This case arises, for example, when one considers the dual formulation for a minimization problem with a moderate number of constraints. The proposed approach is based on using Vaidya’s cutting plane method to minimize with respect to the outer block of variables. This optimization algorithm is especially effective when the dimension of the problem is not very large. An inexact oracle for Vaidya’s method is calculated via an approximate solution of the inner maximization problem, which is solved by the accelerated variance reduced algorithm Katyusha. Thus, we leverage the structure of the problem to achieve fast convergence. Separate complexity bounds for gradients of different components with respect to different variables are obtained in the study. The proposed approach is imposing very mild assumptions about the objective. In particular, neither strong convexity nor smoothness is required with respect to the low-dimensional variable group. The number of steps of the proposed algorithm as well as the arithmetic complexity of each step explicitly depend on the dimensionality of the outer variable, hence the assumption that it is relatively small.

The paper presents various modifications of the Frank–Wolfe algorithm in the equilibrium traffic assignment problem. The Beckman model is used as a model for experiments. In this article, first of all, attention is paid to the choice of the direction of the basic step of the Frank–Wolfe algorithm. Algorithms will be presented: Conjugate Frank–Wolfe (CFW), Bi-conjugate Frank–Wolfe (BFW), Fukushima Frank –Wolfe (FFW). Each modification corresponds to different approaches to the choice of this direction. Some of these modifications are described in previous works of the authors. In this article, following algorithms will be proposed: N-conjugate Frank–Wolfe (NFW), Weighted Fukushima Frank–Wolfe (WFFW). These algorithms are some ideological continuation of the BFW and FFW algorithms. Thus, if the first algorithm used at each iteration the last two directions of the previous iterations to select the next direction conjugate to them, then the proposed algorithm NFW is using more than $N$ previous directions. In the case of Fukushima Frank–Wolfe, the average of several previous directions is taken as the next direction. According to this algorithm, a modification WFFW is proposed, which uses a exponential smoothing from previous directions. For comparative analysis, experiments with various modifications were carried out on several data sets representing urban structures and taken from publicly available sources. The relative gap value was taken as the quality metric. The experimental results showed the advantage of algorithms using the previous directions for step selection over the classic Frank–Wolfe algorithm. In addition, an improvement in efficiency was revealed when using more than two conjugate directions. For example, on various datasets, the modification 3FW showed the best convergence. In addition, the proposed modification WFFW often overtook FFW and CFW, although performed worse than NFW.

The article contains results of modeling a meteor entering dense atmosphere layers using Galerkin’s method and smoother particle hydrodynamics. Numerical simulations were run using experimental data gathered for the Chelyabinsk meteor while varying the meteor material characteristics and its orientation when entering the atmosphere.

We study the influence of fishery on a structured fish population under random changes of habitat conditions. The population parameters correspond to dominant pelagic fish species of Far-Eastern seas of the northwestern part of the Pacific Ocean (pollack, herring, sardine). Similar species inhabit various parts of the Word Ocean. The species body size distribution was chosen as a main population feature. This characteristic is easy to measure and adequately defines main specimen qualities such as age, maturity and other morphological and physiological peculiarities. Environmental fluctuations have a great influence on the individuals in early stages of development and have little influence on the vital activity of mature individuals. The fishery revenue was chosen as an optimality criterion. The main control characteristic is fishing effort. We have chosen quadratic dependence of fishing revenue on the fishing effort according to accepted economic ideas stating that the expenses grow with the production volume. The model study shows that the population structure ensures the increased population stability. The growth and drop out of the individuals’ due to natural mortality smoothens the oscillations of population density arising from the strong influence of the fluctuations of environment on young individuals. The smoothing part is played by diffusion component of the growth processes. The fishery in its turn smooths the fluctuations (including random fluctuations) of the environment and has a substantial impact upon the abundance of fry and the subsequent population dynamics. The optimal time-dependent fishing effort strategy was compared to stationary fishing effort strategy. It is shown that in the case of quickly changing habitat conditions and stochastic dynamics of population replenishment there exists a stationary fishing effort having approximately the same efficiency as an optimal time-dependent fishing effort. This means that a constant or weakly varying fishing effort can be very efficient strategy in terms of revenue.