Package ‘sparseHessianFD’ August 29, 2013 Type Package Title Interface to ACM TOMS Algorithm 636, for computing sparse Hessians. Version 0.1.0 Date 2012-11-15 Author R interface code by Michael Braun Original Fortran code by Thomas F. Coleman, Burton S. Garbow and Jorge J. More. Maintainer Michael Braun URL braunm.scripts.mit.edu Description Computes Hessian of a scalar-valued function, and returns it in sparse Matrix format. The user must supply the objective function, the gradient, and the row and column indices of the non-zero elements of the lower triangle of the Hessian (i.e.,the sparsity structure must be known in advance). The algorithm exploits this sparsity, so Hessians can be computed quickly even when the number of arguments to the objective functions is large. This package is intended to be useful when optimizing objective functions using optimizers than can exploit this sparsity, such as the trustOptim package. The underlying algorithm is ACM TOMS Algorithm 636, written by Coleman, Garbow and More (ACM Transactions on Mathematical Software, 11:4, Dec. 1985). The package also includes functions to sample from, and compute the log density of, a multivariate normal distribution when the precision matrix is sparse. License file LICENSE Depends Rcpp (>= 0.9.6), RcppEigen (>= 0.3.1), Matrix, methods LinkingTo Rcpp, RcppEigen Repository CRAN Date/Publication 2012-11-26 19:09:55 NeedsCompilation yes License_restricts_use yes 1

2

sparseHessianFD-package

R topics documented: sparseHessianFD-package . Coord.to.Pattern.Matrix . . . Coord.to.Sym.Pattern.Matrix Matrix.to.Coord . . . . . . . mvn.sparse . . . . . . . . . new.sparse.hessian.obj . . . sparseHessianObj-class . . . Sym.CSC.to.Matrix . . . . . wrapper functions . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

Index

2 3 4 4 5 6 7 8 9 10

sparseHessianFD-package Given function, gradient, and sparsity structure, returns Hessian in sparse matrix format.

Description This package contains methods to return the Hessian of a function in sparse Matrix format. The user must supply the objective function, the gradient, and the row and column indices of the nonzero elements of the lower triangle of the Hessian (i.e., the sparsity structure must be known in advance). Sparse Hessians occur in many applications, such as log posterior densities of hierarchical models under conditional independence assumptions. This package is intended to be useful when optimizing objective functions with this structure, using optimizers than can exploit this sparsity, such as the trustOptim package. License details are available in the LICENSE file in the source code. Details Package: Type: Version: Date: License: LazyLoad:

sparseHessianFD Package 0.1-0 2012-11-07 See the LICENSE file included in the source code. yes

Author(s) Michael Braun

Coord.to.Pattern.Matrix

3

References Coleman, Thomas F, Burton S Garbow, and Jorge J More. 1985. Software for Estimating Sparse Hessian Matrices. ACM Transaction on Mathematical Software 11 (4) (December): 363–377. Coleman, Thomas F, Burton S Garbow, and Jorge J More. 1985. Algorithm 636: FORTRAN Subroutines for Estimating Sparse Hessian Matrices. ACM Transactions on Mathematical Software 11 (4): 378.

Coord.to.Pattern.Matrix Converts row and column indices to a pattern Matrix object of Matrix class

Description Converts row and column indices to a pattern Matrix object of Matrix class Usage Coord.to.Pattern.Matrix(H, nrows, ncols=nrows) Arguments H

a list containing matrix structure data. See details.

nrows

the number of rows in the matrix.

ncols

the number of columns in the matrix.

Details H is a list with two vectors. iRow Row indices for each of the non-zero elements in the matrix. jCol Column indices for each of the non-zero elements in the matrix. Value An object of Matrix class.

4

Matrix.to.Coord

Coord.to.Sym.Pattern.Matrix Converts row and column indices to a symmetric pattern Matrix object of Matrix class

Description Converts row and column indices to a symmetric pattern Matrix object of Matrix class Usage Coord.to.Sym.Pattern.Matrix(H, nvars) Arguments H

a list containing matrix structure data. See details.

nvars

the number of rows (and columns) in the matrix.

Details H is a list with two vectors. iRow Row indices for each of the non-zero elements in the matrix. jCol Column indices for each of the non-zero elements in the matrix. Only the non-zero elements in the lower triangle should be included. Value An object of Matrix class.

Matrix.to.Coord

Takes an object of class Matrix and returns the row and column indices of the non-zero elements in the lower triangle.

Description Takes an object of class Matrix and returns the row and column indices of the non-zero elements in the lower triangle Usage Matrix.to.Coord(M)

mvn.sparse

5

Arguments M

a symmetric Matrix object.

Value A list suitable for using as the hess.struct argument in get.sparse.hessian. There are three elements: iRow jpntr vals

mvn.sparse

Row indices for each of the non-zero elements in the lower triangle of M. A vector of length nvars+1. jpntr[j] is the index of the element in vals that is the first non-zero element in the jth column of M. The values of the non-zero elements in the lower triangle of M.

Sampling and log density functions for multivariate normal distribution, given mean and the Cholesky decomposition of the precision or covariance matrix

Description Efficient sampling and density calculation from a multivariate normal, when the Cholesky decomposition of the precision matrix is already available. Usage dmvn.sparse(x, mu, L, prec=FALSE) rmvn.sparse(n, mu, L, prec=FALSE) Arguments x n mu L

prec

numeric matrix, where each row is an mvn sample. number of draws mean (numeric vector) Either the lower triangle of the Cholesky decomposition of the covariance or precision (inverse covariance) matrix, stored in a sparse format as defined in the Matrix package. Depends on value of prec. If TRUE, L is the lower triangle of the Cholesky of the precision matrix. If FALSE, L is the lower triangle of the Cholesky decomposition of the covariance matrix.

Details This function uses sparse matrix operations to sample from, or compute the log density of, a multivariate normal distribution The user must compute the Cholesky decomposition first. This must be a LOWER triangle; note that this is not the default behavior of the base chol function, which returns an upper triangle. For stability and efficiency, we recommend passing L as an object of class "triangularMatrix", as defined in the Matrix package.

6

new.sparse.hessian.obj

Value For rmvn.sparse, a matrix with n rows, with each draw in each row. The number of columns is equal to the length of mu. For dmvn.sparse, a vector with a length equal to the number of rows of x, containing the MVN log densities evaluated at each row of x.

new.sparse.hessian.obj Construct new object of class sparseHessianObj

Description Constructs a new object of class sparseHessianObj that can be used to compute the Hessian of a function, given the gradient function and sparsity structure. Usage new.sparse.hessian.obj(x, fn, gr, hs, fd.method=0L, eps=sqrt(.Machine$double.eps), ...) Arguments x

An initial value of the argument to the objective function. Numeric vector.

fn

A scalar-value function that takes x as its first argument.

gr

A function that returns the gradient of fn(x,...), where x is the first argument.

hs

A list of two integer vectors that represents the sparsity structure of the Hessian of fn(x,...). See details.

fd.method

This flag is 1 if using the direct method for computing a finite-differenced Hessian. Otherwise, the indirect method is used. Default value is 0. See details.

eps

The perturbation amount for finite differencing of the gradient to compute the Hessian. Defaults to sqrt(.Machine$double.eps).

...

Additional arguments passed to fn and gr.

Value An object of class sparseHessianObj Hessian structure information You must provide information about the sparsity structure of the Hessian. There are two elements to the hs.struct list: iRow row indices of non-zero entries in the lower triangle of the Hessian jCol column indices of non-zero entries in the lower triangle of the Hessian

sparseHessianObj-class

7

Indexing starts at 1, and should include the diagonal elements, even though it is obvious that the diagonal elements are non-zero. Entries must be ordered by column, and by row within columns. Do not include any entries for the upper triangle. The algorithms used for estimating sparse Hessians using finite differencing are described in Coleman, Thomas F, Burton S Garbow, and Jorge J More. 1985. Software for Estimating Sparse Hessian Matrices. ACM Transaction on Mathematical Software 11 (4) (December): 363-377. This method involves a partitioning and permutation of the Hessian matrix to reduce the number of distinct finite differencing operations of the gradient. There are two methods for computing the partition and permutation: direct and indirect. The direct method tends to require more computation, but may be more accurate. We recommend the indirect method to start, so the default value is 0.

sparseHessianObj-class Class to store function, gradient and Hessian structure information

Description The class "sparseHessianObj" contains an external pointer to a C++ class that contains the methods that are used to compute the sparse Hessian. It can be constructed directly using "new", or through the "new.sparse.hessian.obj" function. The latter approach includes some additional argument checking. Slots pointer: Object of class "Rcpp:::externalptr" Constructing objects Objects of the class can be constructed directly by: new("sparseHessianObj", nvars, fn, gr, hs.struct, fd.method, eps) nvars The length of the function argument vector x fn A function that takes a single argument and returns the value of the objective function, evaluated at x. gr A function that takes a single argument and returns the gradient of the objective function, evaluated at x. hs.struct A list of two integer vectors that describe the sparsity structure of the Hessian. See details. fd.method An integer, 0 for the indirect method and 1 for the direct method, as described in Coleman, et.al (1985). The indirect method is somewhat faster, and suitable for most applications. eps The perturbation about for finite differencing. Typically sqrt(.Machine$double.eps).

8

Sym.CSC.to.Matrix

Methods $fn(x) signature(x="sparseHessianObj"): returns fn(x) $gr(x) signature(x="sparseHessianObj"): returns gr(x) $fngr(x) signature(x="sparseHessianObj"): returns list of fn(x) and gr(x) $hessian(x) signature(x="sparseHessianObj"): returns hessian(x) as "dgCMatrix" $all(x) signature(x="sparseHessianObj"): returns list of fn(x), gr(x) and hessian See Also new.sparse.hessian.obj

Converts symmetric Hessian stored in CSC format to an object of Matrix class

Sym.CSC.to.Matrix

Description Converts symmetric Hessian stored in CSC format to an object of Matrix class. Usage Sym.CSC.to.Matrix(H, nvars) Arguments H

a list containing Hessian data. See details.

nvars

the number of rows (and columns) in the matrix.

Details H is a list with three elements. iRow Row indices for each of the non-zero elements in the lower triangle of H jpntr A vector of length nvars+1. jpntr[j] is the index of the element in vals that is the first nonzero element in the jth column of the matrix. vals The values of the non-zero elements in the lower triangle of the matrix. Value An object of Matrix class.

wrapper functions

9

Compute function, gradient and Hessian from objects of class sparseHessianObj.

wrapper functions

Description Wrapper functions to call methods of an object of class sparseHessianObj. Usage get.fn(x, obj) get.gr(x, obj) get.fngr(x, obj) get.hessian(x, obj) Arguments x

Value at which fn, gr or the hessian is evaluated.

obj

Object of class sparseHessianObj that contains all of the function, gradient and Hessian information.

Index ∗Topic package sparseHessianFD-package, 2 $,sparseHessianObj-method (sparseHessianObj-class), 7 Coord.to.Pattern.Matrix, 3 Coord.to.Sym.Pattern.Matrix, 4 dmvn.sparse (mvn.sparse), 5 get.fn (wrapper functions), 9 get.fngr (wrapper functions), 9 get.gr (wrapper functions), 9 get.hessian (wrapper functions), 9 Matrix.to.Coord, 4 mvn.sparse, 5 new, sparseHessianObj-method (sparseHessianObj-class), 7 new.sparse.hessian.obj, 6 rmvn.sparse (mvn.sparse), 5 sparseHessianFD (sparseHessianFD-package), 2 sparseHessianFD-package, 2 sparseHessianObj-class, 7 Sym.CSC.to.Matrix, 8 wrapper functions, 9

10