Результаты поиска по 'component descent':
Найдено статей: 2
  1. Vostrikov D.D., Konin G.O., Lobanov A.V., Matyukhin V.V.
    Influence of the mantissa finiteness on the accuracy of gradient-free optimization methods
    Computer Research and Modeling, 2023, v. 15, no. 2, pp. 259-280

    Gradient-free optimization methods or zeroth-order methods are widely used in training neural networks, reinforcement learning, as well as in industrial tasks where only the values of a function at a point are available (working with non-analytical functions). In particular, the method of error back propagation in PyTorch works exactly on this principle. There is a well-known fact that computer calculations use heuristics of floating-point numbers, and because of this, the problem of finiteness of the mantissa arises.

    In this paper, firstly, we reviewed the most popular methods of gradient approximation: Finite forward/central difference (FFD/FCD), Forward/Central wise component (FWC/CWC), Forward/Central randomization on $l_2$ sphere (FSSG2/CFFG2); secondly, we described current theoretical representations of the noise introduced by the inaccuracy of calculating the function at a point: adversarial noise, random noise; thirdly, we conducted a series of experiments on frequently encountered classes of problems, such as quadratic problem, logistic regression, SVM, to try to determine whether the real nature of machine noise corresponds to the existing theory. It turned out that in reality (at least for those classes of problems that were considered in this paper), machine noise turned out to be something between adversarial noise and random, and therefore the current theory about the influence of the mantissa limb on the search for the optimum in gradient-free optimization problems requires some adjustment.

    Keywords: mantissa finiteness, gradient-free optimization, gradient approximation, gradient descent, quadratic problem, logistic regression.
  2. Pletnev N.V., Matyukhin V.V.
    On the modification of the method of component descent for solving some inverse problems of mathematical physics
    Computer Research and Modeling, 2023, v. 15, no. 2, pp. 301-316

    The article is devoted to solving ill-posed problems of mathematical physics for elliptic and parabolic equations, such as the Cauchy problem for the Helmholtz equation and the retrospective Cauchy problem for the heat equation with constant coefficients. These problems are reduced to problems of convex optimization in Hilbert space. The gradients of the corresponding functionals are calculated approximately by solving two well-posed problems. A new method is proposed for solving the optimization problems under study, it is component-by-component descent in the basis of eigenfunctions of a self-adjoint operator associated with the problem. If it was possible to calculate the gradient exactly, this method would give an arbitrarily exact solution of the problem, depending on the number of considered elements of the basis. In real cases, the inaccuracy of calculations leads to a violation of monotonicity, which requires the use of restarts and limits the achievable quality. The paper presents the results of experiments confirming the effectiveness of the constructed method. It is determined that the new approach is superior to approaches based on the use of gradient optimization methods: it allows to achieve better quality of solution with significantly less computational resources. It is assumed that the constructed method can be generalized to other problems.

    Keywords: inverse problems, convex optimization, optimization in a Hilbert space, first order methods, component descent, imprecise oracle.

Indexed in Scopus

Full-text version of the journal is also available on the web site of the scientific electronic library eLIBRARY.RU

The journal is included in the Russian Science Citation Index

The journal is included in the RSCI

International Interdisciplinary Conference "Mathematics. Computing. Education"

Copyright © 2009–2024 Institute of Computer Science