Sparse Hessian

@(@\newcommand{\W}[1]{ \; #1 \; } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\B}[1]{ {\bf #1} } \newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} } \newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} } \newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} } \newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }@)@This is cppad-20221105 documentation. Here is a link to its current documentation . Sparse Hessian
Syntax

hes = f.SparseHessian(x, w)

hes = f.SparseHessian(x, w, p)

n_sweep = f.SparseHessian(x, w, p, row, col, hes, work)

Purpose
We use @(@ n @)@ for the domain size, and @(@ m @)@ for the range size of f . We use @(@ F : \B{R}^n \rightarrow \B{R}^m @)@ do denote the AD function corresponding to f . The syntax above sets hes to the Hessian @[@ H(x) = \dpow{2}{x} \sum_{i=1}^m w_i F_i (x) @]@ This routine takes advantage of the sparsity of the Hessian in order to reduce the amount of computation necessary. If row and col are present, it also takes advantage of the reduced set of elements of the Hessian that need to be computed. One can use speed tests (e.g. speed_test ) to verify that results are computed faster than when using the routine Hessian .

f
The object f has prototype



    ADFun<Base> f

Note that the ADFun object f is not const (see Uses Forward below).

x
The argument x has prototype



    const BaseVector& x

(see BaseVector below) and its size must be equal to n , the dimension of the domain space for f . It specifies that point at which to evaluate the Hessian.

w
The argument w has prototype



    const BaseVector& w

and size @(@ m @)@. It specifies the value of @(@ w_i @)@ in the expression for hes . The more components of @(@ w @)@ that are identically zero, the more sparse the resulting Hessian may be (and hence the more efficient the calculation of hes may be).

p
The argument p is optional and has prototype



    const SetVector& p

(see SetVector below) If it has elements of type bool, its size is @(@ n * n @)@. If it has elements of type std::set<size_t>, its size is @(@ n @)@ and all its set elements are between zero and @(@ n - 1 @)@. It specifies a sparsity pattern for the Hessian @(@ H(x) @)@.

Purpose
If this sparsity pattern does not change between calls to SparseHessian , it should be faster to calculate p once and pass this argument to SparseHessian . If you specify p , CppAD will use the same type of sparsity representation (vectors of bool or vectors of std::set<size_t>) for its internal calculations. Otherwise, the representation for the internal calculations is unspecified.

work
If you specify work in the calling sequence, it is not necessary to keep the sparsity pattern; see the heading p under the work description.

Column Subset
If the arguments row and col are present, and color_method is cppad.general or cppad.symmetric, it is not necessary to compute the entire sparsity pattern. Only the following subset of column values will matter:



    { col[k] : k = 0 , ... , K-1 }

.

row, col
The arguments row and col are optional and have prototype



    const SizeVector& row

    const SizeVector& col

(see SizeVector below). They specify which rows and columns of @(@ H (x) @)@ are returned and in what order. We use @(@ K @)@ to denote the value hes.size() which must also equal the size of row and col . Furthermore, for @(@ k = 0 , \ldots , K-1 @)@, it must hold that @(@ row[k] < n @)@ and @(@ col[k] < n @)@. In addition, all of the @(@ (row[k], col[k]) @)@ pairs must correspond to a true value in the sparsity pattern p .

hes
The result hes has prototype



    BaseVector hes

In the case where row and col are not present, the size of hes is @(@ n * n @)@ and its size is @(@ n * n @)@. In this case, for @(@ i = 0 , \ldots , n - 1 @)@ and @(@ ell = 0 , \ldots , n - 1 @)@ @[@ hes [ j * n + \ell ] = \DD{ w^{\rm T} F }{ x_j }{ x_\ell } ( x ) @]@

In the case where the arguments row and col are present, we use @(@ K @)@ to denote the size of hes . The input value of its elements does not matter. Upon return, for @(@ k = 0 , \ldots , K - 1 @)@, @[@ hes [ k ] = \DD{ w^{\rm T} F }{ x_j }{ x_\ell } (x) \; , \; \; {\rm where} \; j = row[k] \; {\rm and } \; \ell = col[k] @]@

work
If this argument is present, it has prototype



    sparse_hessian_work& work

This object can only be used with the routines SparseHessian. During its the first use, information is stored in work . This is used to reduce the work done by future calls to SparseHessian with the same f , p , row , and col . If a future call is made where any of these values have changed, you must first call work.clear() to inform CppAD that this information needs to be recomputed.

color_method
The coloring algorithm determines which rows and columns can be computed during the same sweep. This field has prototype



    std::string work.color_method

This value only matters on the first call to sparse_hessian that follows the work constructor or a call to work.clear() .





"cppad.symmetric"

This is the default coloring method (after a constructor or clear()). It takes advantage of the fact that the Hessian matrix is symmetric to find a coloring that requires fewer sweeps .





"cppad.general"

This is the same as the "cppad" method for the sparse_jacobian calculation.





"colpack.symmetric"

This method requires that colpack_prefix was specified on the cmake command line. It also takes advantage of the fact that the Hessian matrix is symmetric.





"colpack.general"

This is the same as the "colpack" method for the sparse_jacobian calculation.

colpack.star Deprecated 2017-06-01
The colpack.star method is deprecated. It is the same as the colpack.symmetric which should be used instead.

p
If work is present, and it is not the first call after its construction or a clear, the sparsity pattern p is not used. This enables one to free the sparsity pattern and still compute corresponding sparse Hessians.

n_sweep
The return value n_sweep has prototype



    size_t n_sweep

It is the number of first order forward sweeps used to compute the requested Hessian values. Each first forward sweep is followed by a second order reverse sweep so it is also the number of reverse sweeps. This is proportional to the total work that SparseHessian does, not counting the zero order forward sweep, or the work to combine multiple columns into a single forward-reverse sweep pair.

BaseVector
The type BaseVector must be a SimpleVector class with elements of type Base . The routine CheckSimpleVector will generate an error message if this is not the case.

SetVector
The type SetVector must be a SimpleVector class with elements of type bool or std::set<size_t>; see sparsity pattern for a discussion of the difference. The routine CheckSimpleVector will generate an error message if this is not the case.

Restrictions
If SetVector has elements of std::set<size_t>, then p[i] must return a reference (not a copy) to the corresponding set. According to section 26.3.2.3 of the 1998 C++ standard, std::valarray< std::set<size_t> > does not satisfy this condition.

SizeVector
The type SizeVector must be a SimpleVector class with elements of type size_t. The routine CheckSimpleVector will generate an error message if this is not the case.

Uses Forward
After each call to Forward , the object f contains the corresponding Taylor coefficients . After a call to any of the sparse Hessian routines, the zero order Taylor coefficients correspond to f.Forward(0, x) and the other coefficients are unspecified.

Example
The routine sparse_hessian.cpp is examples and tests of sparse_hessian. It return true, if it succeeds and false otherwise.

Subset Hessian
The routine sub_sparse_hes.cpp is an example and test that compute a sparse Hessian for a subset of the variables. It returns true, for success, and false otherwise.

Input File: include/cppad/core/sparse_hessian.hpp