All issues

Finding the global minimum of a nonconvex function is one of the key and most difficult problems of the modern optimization. In this paper we consider special classes of nonconvex problems which have a clear and distinct global minimum.

In the first part of the paper we consider two classes of «good» nonconvex functions, which can be bounded below and above by a parabolic function. This class of problems has not been widely studied in the literature, although it is rather interesting from an applied point of view. Moreover, for such problems first-order and higher-order methods may be completely ineffective in finding a global minimum. This is due to the fact that the function may oscillate heavily or may be very noisy. Therefore, our new methods use only zero-order information and are based on grid search. The size and fineness of this grid, and hence the guarantee of convergence speed and oracle complexity, depend on the «goodness» of the problem. In particular, we show that if the function is bounded by fairly close parabolic functions, then the complexity is independent of the dimension of the problem. We show that our new methods converge with a linear convergence rate $\log(1/\varepsilon)$ to a global minimum on the cube.

In the second part of the paper, we consider the nonconvex optimization problem from a different angle. We assume that the target minimizing function is the sum of the convex quadratic problem and a nonconvex «noise» function proportional to the distance to the global solution. Considering functions with such noise assumptions for zero-order methods is new in the literature. For such a problem, we use the classical gradient-free approach with gradient approximation through finite differences. We show how the convergence analysis for our problems can be reduced to the standard analysis for convex optimization problems. In particular, we achieve a linear convergence rate for such problems as well.

Experimental results confirm the efficiency and practical applicability of all the obtained methods.

The paper studies a multidimensional convection-diffusion equation with variable coefficients and a nonclassical boundary condition. Two cases are considered: in the first case, the first boundary condition contains the integral of the unknown function with respect to the integration variable $x_\alpha^{}$, and in the second case, the integral of the unknown function with respect to the integration variable $\tau$, denoting the memory effect. Similar problems arise when studying the transport of impurities along the riverbed. For an approximate solution of the problem posed, a locally one-dimensional difference scheme by A.A. Samarskii with order of approximation $O(h^2+\tau)$. In view of the fact that the equation contains the first derivative of the unknown function with respect to the spatial variable $x_\alpha^{}$, the wellknown method proposed by A.A. Samarskii in constructing a monotonic scheme of the second order of accuracy in $h_\alpha^{}$ for a general parabolic type equation containing one-sided derivatives taking into account the sign of $r_\alpha^{}(x,t)$. To increase the boundary conditions of the third kind to the second order of accuracy in $h_\alpha^{}$, we used the equation, on the assumption that it is also valid at the boundaries. The study of the uniqueness and stability of the solution was carried out using the method of energy inequalities. A priori estimates are obtained for the solution of the difference problem in the $L_2^{}$-norm, which implies the uniqueness of the solution, the continuous and uniform dependence of the solution of the difference problem on the input data, and the convergence of the solution of the locally onedimensional difference scheme to the solution of the original differential problem in the $L_2^{}$-norm with speed equal to the order of approximation of the difference scheme. For a two-dimensional problem, a numerical solution algorithm is constructed.

A variant of the inverse method of characteristics (IMH) is presented, in whose algorithm an additional fractional time step is introduced, which makes it possible to increase the accuracy of calculations due to a more accurate approximation of the characteristics. The calculation formulas of the modified method for the equations of the one-velocity model of a gas-liquid mixture are given, with the help of which one-dimensional and also flat test problems with self-similar solutions are calculated. When solving multidimensional problems, the original system of equations is split into a number of one-dimensional subsystems, for the calculation of which the inverse method of characteristics with a fractional time step is used. Using the proposed method, the following were calculated: the one-dimensional problem of the decay of an arbitrary discontinuity in a dispersed medium; a twodimensional problem of the interaction of a homogeneous gas-liquid flow with an obstacle with an attached shock wave, as well as a flow with a centered rarefaction wave. The results of numerical calculations of these problems are compared with self-similar solutions and their satisfactory agreement is noted. On the example of the Riemann problem with a shock wave, a comparison is made with a number of conservative, non-conservative, first and higher orders of accuracy schemes, from which, in particular, it follows that the presented calculation method, i. e. MIMC, quite competitive. Despite the fact that the application of MIMC requires many times more time than the original inverse method of characteristics (IMC), calculations can be carried out with an increased time step and, in some cases, more accurate results can be obtained. It is noted that the method with a fractional time step has advantages over the IMC in cases where the characteristics of the system are significantly curvilinear. For this reason, the use of MIMC, for example, for the Euler equations is inappropriate, since for the latter the characteristics within the time step differ little from straight lines.

Gradient-free optimization methods or zeroth-order methods are widely used in training neural networks, reinforcement learning, as well as in industrial tasks where only the values of a function at a point are available (working with non-analytical functions). In particular, the method of error back propagation in PyTorch works exactly on this principle. There is a well-known fact that computer calculations use heuristics of floating-point numbers, and because of this, the problem of finiteness of the mantissa arises.

In this paper, firstly, we reviewed the most popular methods of gradient approximation: Finite forward/central difference (FFD/FCD), Forward/Central wise component (FWC/CWC), Forward/Central randomization on $l_2$ sphere (FSSG2/CFFG2); secondly, we described current theoretical representations of the noise introduced by the inaccuracy of calculating the function at a point: adversarial noise, random noise; thirdly, we conducted a series of experiments on frequently encountered classes of problems, such as quadratic problem, logistic regression, SVM, to try to determine whether the real nature of machine noise corresponds to the existing theory. It turned out that in reality (at least for those classes of problems that were considered in this paper), machine noise turned out to be something between adversarial noise and random, and therefore the current theory about the influence of the mantissa limb on the search for the optimum in gradient-free optimization problems requires some adjustment.

A nonlinear oscillatory system described by ordinary differential equations with variable coefficients is considered, in which terms that are linearly dependent on coordinates, velocities and accelerations are explicitly distinguished; nonlinear terms are written as implicit functions of these variables. For the numerical solution of the initial problem described by such a system of differential equations, the one-step Galerkin method is used. At the integration step, unknown functions are represented as a sum of linear functions satisfying the initial conditions and several given correction functions in the form of polynomials of the second and higher degrees with unknown coefficients. The differential equations at the step are satisfied approximately by the Galerkin method on a system of corrective functions. Algebraic equations with nonlinear terms are obtained, which are solved by iteration at each step. From the solution at the end of each step, the initial conditions for the next step are determined.

The corrective functions are taken the same for all steps. In general, 4 or 5 correction functions are used for calculations over long time intervals: in the first set — basic power functions from the 2nd to the 4th or 5th degrees; in the second set — orthogonal power polynomials formed from basic functions; in the third set — special linear-independent polynomials with finite conditions that simplify the “docking” of solutions in the following steps.

Using two examples of calculating nonlinear oscillations of systems with one and two degrees of freedom, numerical studies of the accuracy of the numerical solution of initial problems at various time intervals using the Galerkin method using the specified sets of power-law correction functions are performed. The results obtained by the Galerkin method and the Adams and Runge –Kutta methods of the fourth order are compared. It is shown that the Galerkin method can obtain reliable results at significantly longer time intervals than the Adams and Runge – Kutta methods.

The paper deals with a heat wave motion problem for a degenerate second-order nonlinear parabolic equation with power nonlinearity. The considered boundary condition specifies in a plane the motion equation of the circular zero front of the heat wave. A new numerical-analytical algorithm for solving the problem is proposed. A solution is constructed stepby- step in time using difference time discretization. At each time step, a boundary value problem for the Poisson equation corresponding to the original equation at a fixed time is considered. This problem is, in fact, an inverse Cauchy problem in the domain whose initial boundary is free of boundary conditions and two boundary conditions (Neumann and Dirichlet) are specified on a current boundary (heat wave). A solution of this problem is constructed as the sum of a particular solution to the nonhomogeneous Poisson equation and a solution to the corresponding Laplace equation satisfying the boundary conditions. Since the inhomogeneity depends on the desired function and its derivatives, an iterative solution procedure is used. The particular solution is sought by the collocation method using inhomogeneity expansion in radial basis functions. The inverse Cauchy problem for the Laplace equation is solved by the null field method as applied to a circular domain with a circular hole. This method is used for the first time to solve such problem. The calculation algorithm is optimized by parallelizing the computations. The parallelization of the computations allows us to realize effectively the algorithm on high performance computing servers. The algorithm is implemented as a program, which is parallelized by using the OpenMP standard for the C++ language, suitable for calculations with parallel cycles. The effectiveness of the algorithm and the robustness of the program are tested by the comparison of the calculation results with the known exact solution as well as with the numerical solution obtained earlier by the authors with the use of the boundary element method. The implemented computational experiment shows good convergence of the iteration processes and higher calculation accuracy of the proposed new algorithm than of the previously developed one. The solution analysis allows us to select the radial basis functions which are most suitable for the proposed algorithm.

This paper investigates neural network models and polynomial models based on Chebyshev polynomials for interference compensation. It is shown that the neural network model provides compensation for parasitic interference without the need for parameter tuning, unlike the polynomial model, which requires the selection of optimal delays. The L-BFGS method is applied to both architectures, achieving a compensation level comparable to the LS solution for the polynomial model, with an NMSE result of −23.59 dB and requiring fewer than 2000 iterations, confirming its high efficiency. Additionally, due to the strong generalization ability of neural network architectures, the first-order method for neural networks demonstrates faster convergence compared to the polynomial model. In 20 000 iterations, the neural network model achieves a 0.44 dB improvement in compensation level compared to the polynomial model. In contrast, the polynomial model can only achieve high compensation levels with optimal first-order method parameter tuning, highlighting one of the key advantages of neural network models.

This article is dedicated to use of S-spline theory for solving equations in partial derivatives. For example, we consider solution of the Poisson equation. S-spline — is a piecewise-polynomial function. Its coefficients are defined by two states. The first part of coefficients are defined by smoothness of the spline. The second coefficients are determined by least-squares method. According to order of considered polynomial and number of conditions of first and second type we get S-splines with different properties. At this moment we have investigated order 3 S-splines of class C¹ and order 5 S-splines of class C² (they meet conditions of smoothness of order 1 and 2 respectively). We will consider how the order 3 S-splines of class C¹ can be applied for solving equation of Poisson on circle and other areas.

Newton and quasi-Newton methods of absolute optimization based on Cholesky factorization with adaptive step and finite difference approximation of the first and the second derivatives. In order to raise effectiveness of the quasi-Newton methods a modified version of Cholesky decomposition of quasi-Newton matrix is suggested. It solves the problem of step scaling while descending, allows approximation by non-quadratic functions, and integration with confidential neighborhood method. An approach to raise Newton methods effectiveness with finite difference approximation of the first and second derivatives is offered. The results of numerical research of algorithm effectiveness are shown.

WENO schemes (weighted, essentially non oscillating) are currently having a wide range of applications as approximate high order schemes for discontinuous solutions of partial differential equations. These schemes are used for direct numerical simulation (DNS) and large eddy simmulation in the gas dynamic problems, problems for DNS in MHD and even neutron kinetics. This work is dedicated to clarify some characteristics of WENO schemes and numerical simulation of specific tasks. Results of the simulations can be used to clarify the field of application of these schemes. The first part of the work contained proofs of the approximation properties, stability and convergence of WENO5, WENO7, WENO9, WENO11 and WENO13 schemes. In the second part of the work the modified wave number analysis is conducted that allows to conclude the dispersion and dissipative properties of schemes. Further, a numerical simulation of a number of specific problems for hyperbolic equations is conducted, namely for advection equations (one-dimensional and two-dimensional), Hopf equation, Burgers equation (with low dissipation) and equations of non viscous gas dynamics (onedimensional and two-dimensional). For each problem that is implying a smooth solution, the practical calculation of the order of approximation via Runge method is performed. The influence of a time step on nonlinear properties of the schemes is analyzed experimentally in all problems and cross checked with the first part of the paper. In particular, the advection equations of a discontinuous function and Hopf equations show that the failure of the recommendations from the first part of the paper leads first to an increase in total variation of the solution and then the approximation is decreased by the non-linear dissipative mechanics of the schemes. Dissipation of randomly distributed initial conditions in a periodic domain for one-dimensional Burgers equation is conducted and a comparison with the spectral method is performed. It is concluded that the WENO7–WENO13 schemes are suitable for direct numerical simulation of turbulence. At the end we demonstrate the possibility of the schemes to be used in solution of initial-boundary value problems for equations of non viscous gas dynamics: Rayleigh–Taylor instability and the reflection of the shock wave from a wedge with the formation a complex configuration of shock waves and discontinuities.