Prev

Next

Index-> contents reference index search external

Up-> CppAD Theory ReverseTheory

Headings-> Taylor Notation Binary Operators ---..Addition ---..Subtraction ---..Multiplication ---..Division Standard Math Functions Contents

@(@\newcommand{\W}[1]{ \; #1 \; } \newcommand{\R}[1]{ {\rm #1} } \newcommand{\B}[1]{ {\bf #1} } \newcommand{\D}[2]{ \frac{\partial #1}{\partial #2} } \newcommand{\DD}[3]{ \frac{\partial^2 #1}{\partial #2 \partial #3} } \newcommand{\Dpow}[2]{ \frac{\partial^{#1}}{\partial {#2}^{#1}} } \newcommand{\dpow}[2]{ \frac{ {\rm d}^{#1}}{{\rm d}\, {#2}^{#1}} }@)@ This is cppad-20221105 documentation. Here is a link to its current documentation . The Theory of Reverse Mode
Taylor Notation
In Taylor notation, each variable corresponds to a function of a single argument which we denote by t (see Section 10.2 of Evaluating Derivatives ). Here and below @(@ X(t) @)@, @(@ Y(t) @)@, and Z(t) are scalar valued functions and the corresponding p-th order Taylor coefficients row vectors are @(@ x @)@, @(@ y @)@ and @(@ z @)@; i.e., @[@ \begin{array}{lcr} X(t) & = & x^{(0)} + x^{(1)} * t + \cdots + x^{(p)} * t^p + O( t^{p+1} ) \\ Y(t) & = & y^{(0)} + y^{(1)} * t + \cdots + y^{(p)} * t^p + O( t^{p+1} ) \\ Z(t) & = & z^{(0)} + z^{(1)} * t + \cdots + z^{(p)} * t^p + O( t^{p+1} ) \end{array} @]@ For the purposes of this discussion, we are given the p-th order Taylor coefficient row vectors @(@ x @)@, @(@ y @)@, and @(@ z @)@. In addition, we are given the partial derivatives of a scalar valued function @[@ G ( z^{(j)} , \ldots , z^{(0)}, x, y) @]@ We need to compute the partial derivatives of the scalar valued function @[@ H ( z^{(j-1)} , \ldots , z^{(0)}, x, y) = G ( z^{(j)}, z^{(j-1)} , \ldots , z^{(0)}, x , y ) @]@ where @(@ z^{(j)} @)@ is expressed as a function of the j-1-th order Taylor coefficient row vector for @(@ Z @)@ and the vectors @(@ x @)@, @(@ y @)@; i.e., @(@ z^{(j)} @)@ above is a shorthand for @[@ z^{(j)} ( z^{(j-1)} , \ldots , z^{(0)}, x, y ) @]@ If we do not provide a formula for a partial derivative of @(@ H @)@, then that partial derivative has the same value as for the function @(@ G @)@.

Binary Operators

Addition
The forward mode formula for addition is @[@ z^{(j)} = x^{(j)} + y^{(j)} @]@ If follows that for @(@ k = 0 , \ldots , j @)@ and @(@ l = 0 , \ldots , j-1 @)@ @[@ \begin{array}{rcl} \D{H}{ x^{(k)} } & = & \D{G}{ x^{(k)} } + \D{G}{ z^{(k)} } \\ \\ \D{H}{ y^{(k)} } & = & \D{G}{ y^{(k)} } + \D{G}{ z^{(k)} } \\ \D{H}{ z^{(l)} } & = & \D{G}{ z^{(l)} } \end{array} @]@

Subtraction
The forward mode formula for subtraction is @[@ z^{(j)} = x^{(j)} - y^{(j)} @]@ If follows that for @(@ k = 0 , \ldots , j @)@ @[@ \begin{array}{rcl} \D{H}{ x^{(k)} } & = & \D{G}{ x^{(k)} } - \D{G}{ z^{(k)} } \\ \\ \D{H}{ y^{(k)} } & = & \D{G}{ y^{(k)} } - \D{G}{ z^{(k)} } \end{array} @]@

Multiplication
The forward mode formula for multiplication is @[@ z^{(j)} = \sum_{k=0}^j x^{(j-k)} * y^{(k)} @]@ If follows that for @(@ k = 0 , \ldots , j @)@ and @(@ l = 0 , \ldots , j-1 @)@ @[@ \begin{array}{rcl} \D{H}{ x^{(j-k)} } & = & \D{G}{ x^{(j-k)} } + \sum_{k=0}^j \D{G}{ z^{(j)} } y^{(k)} \\ \D{H}{ y^{(k)} } & = & \D{G}{ y^{(k)} } + \sum_{k=0}^j \D{G}{ z^{(j)} } x^{(j-k)} \end{array} @]@

Division
The forward mode formula for division is @[@ z^{(j)} = \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right) @]@ If follows that for @(@ k = 1 , \ldots , j @)@ @[@ \begin{array}{rcl} \D{H}{ x^{(j)} } & = & \D{G}{ x^{(j)} } + \D{G}{ z^{(j)} } \frac{1}{y^{(0)}} \\ \D{H}{ z^{(j-k)} } & = & \D{G}{ z^{(j-k)} } - \D{G}{ z^{(j)} } \frac{1}{y^{(0)}} y^{(k)} \\ \D{H}{ y^{(k)} } & = & \D{G}{ y^{(k)} } - \D{G}{ z^{(j)} } \frac{1}{y^{(0)}} z^{(j-k)} \\ \D{H}{ y^{(0)} } & = & \D{G}{ y^{(0)} } - \D{G}{ z^{(j)} } \frac{1}{y^{(0)}} \frac{1}{y^{(0)}} \left( x^{(j)} - \sum_{k=1}^j z^{(j-k)} y^{(k)} \right) \\ & = & \D{G}{ y^{(0)} } - \D{G}{ z^{(j)} } \frac{1}{y^{(0)}} z^{(j)} \end{array} @]@

Standard Math Functions
The standard math functions have only one argument. Hence we are given the partial derivatives of a scalar valued function @[@ G ( z^{(j)} , \ldots , z^{(0)}, x) @]@ We need to compute the partial derivatives of the scalar valued function @[@ H ( z^{(j-1)} , \ldots , z^{(0)}, x) = G ( z^{(j)}, z^{(j-1)} , \ldots , z^{(0)}, x) @]@ where @(@ z^{(j)} @)@ is expressed as a function of the j-1-th order Taylor coefficient row vector for @(@ Z @)@ and the vector @(@ x @)@; i.e., @(@ z^{(j)} @)@ above is a shorthand for @[@ z^{(j)} ( z^{(j-1)} , \ldots , z^{(0)}, x ) @]@

Contents

exp_reverse	Exponential Function Reverse Mode Theory
log_reverse	Logarithm Function Reverse Mode Theory
sqrt_reverse	Square Root Function Reverse Mode Theory
sin_cos_reverse	Trigonometric and Hyperbolic Sine and Cosine Reverse Theory
atan_reverse	Inverse Tangent and Hyperbolic Tangent Reverse Mode Theory
asin_reverse	Inverse Sine and Hyperbolic Sine Reverse Mode Theory
acos_reverse	Inverse Cosine and Hyperbolic Cosine Reverse Mode Theory
tan_reverse	Tangent and Hyperbolic Tangent Reverse Mode Theory
erf_reverse	Error Function Reverse Mode Theory
pow_reverse	Power Function Reverse Mode Theory

---

Input File: omh/theory/reverse_theory.omh