#+TITLE: Likelihood, first and second order derivatives in a LVM
#+AUTHOR: Brice Ozenne
#+DATE: 

In this document, we show the expression of the likelihood, its first
two derivatives, the information matrix, and the first derivative of
the information matrix.

* Likelihood

At the individual level, the measurement and structural models can be written:
#+BEGIN_EXPORT LaTeX
\begin{align*}
\VY_i &= \nu + \Veta_i \Lambda + \VX_i K + \Vvarepsilon_i \\
\Veta_i &= \alpha + \Veta_i B + \VX_i \Gamma + \boldsymbol{\zeta}_i 
\end{align*}
\begin{tabular}{lll}
with & \(\Sigma_{\epsilon}\)   &the variance-covariance matrix of the residuals \(\Vvarepsilon_i\)\\
     & \(\Sigma_{\zeta}\) & the variance-covariance matrix of the residuals \(\boldsymbol{\zeta}_i\). \\
\end{tabular}
#+END_EXPORT

\bigskip

By combining the previous equations, we can get an expression for
\(\VY_i\) that does not depend on \(\Veta_i\):
#+BEGIN_EXPORT LaTeX
\begin{align*}
\VY_i &= \nu + \left(\boldsymbol{\zeta}_i + \alpha + \VX_i \Gamma \right) (I-B)^{-1} \Lambda + \VX_i K + \Vvarepsilon_i 
\end{align*}
#+END_EXPORT
Since \(\Var[Ax] = A \Var[x] \trans{A}\) we have \(\Var[xA] =
\trans{A} \Var[x] A\), we have the following expressions for the
conditional mean and variance of \(\VY_i\):
#+BEGIN_EXPORT LaTeX
\begin{align*}
 \Vmu(\Vparam,\VX_i) &= E[\VY_i|\VX_i] = \nu + (\alpha + \VX_i \Gamma) (1-B)^{-1} \Lambda + \VX_i K \\
\Omega(\Vparam) &= Var[\VY_i|\VX_i] = \Lambda^t (1-B)^{-t}  \Sigma_{\zeta} (1-B)^{-1} \Lambda + \Sigma_{\varepsilon} 
\end{align*}
#+END_EXPORT

\bigskip
 
where \(\Vparam\) is the collection of all parameters. The
log-likelihood can be written:
 #+BEGIN_EXPORT LaTeX
\begin{align*}
l(\Vparam|\VY,\VX) &= \sum_{i=1}^n l(\Vparam|\VY_i,\VX_i) \\
&= \sum_{i=1}^{n} - \frac{p}{2} log(2\pi) - \frac{1}{2} log|\Omega(\Vparam)| - \frac{1}{2} (\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))}
\end{align*}
 #+END_EXPORT

* Partial derivative for the conditional mean and variance

In the following, we denote by \(\delta_{\sigma \in \Sigma}\) the
indicator matrix taking value 1 at the position of \(\sigma\) in the
matrix \(\Sigma\). For instance:
#+BEGIN_EXPORT latex
\begin{align*}
\Sigma =
\begin{bmatrix}
 \sigma_{1,1} & \sigma_{1,2} & \sigma_{1,3} \\
 \sigma_{1,2} & \sigma_{2,2} & \sigma_{2,3} \\
 \sigma_{1,3} & \sigma_{2,3} & \sigma_{3,3} \\
\end{bmatrix}
 \qquad 
\delta_{\sigma_{1,2} \in \Sigma} =
\begin{bmatrix}
0 & 1 & 0 \\
1 & 0 & 0 \\
0 & 0 & 0 \\
\end{bmatrix}
\end{align*}
#+END_EXPORT
The same goes for \(\delta_{\lambda \in \Lambda}\), \(\delta_{b \in
B}\), and \(\delta_{\psi \in \Psi}\). 

\bigskip

First order derivatives:
#+BEGIN_EXPORT LaTeX
\begin{align*}
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \nu} &= 1 \\
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial K} &= \VX_i \\
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \alpha} &= (1-B)^{-1}\Lambda \\
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \Gamma} &= \VX_i(1-B)^{-1}\Lambda \\
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \lambda} &= (\alpha + \VX_i \Gamma)(1-B)^{-1}\delta_{\lambda \in \Lambda} \\
 \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial b} &= (\alpha + \VX_i \Gamma)(1-B)^{-1}\delta_{b \in B}(1-B)^{-1}\Lambda \\
 &\\
 \frac{\partial \Omega(\Vparam)}{\partial \psi} &= \Lambda^t (1-B)^{-t} \delta_{\psi \in \Psi} (1-B)^{-1} \Lambda \\
 \frac{\partial \Omega(\Vparam)}{\partial \sigma} &= \delta_{\sigma \in \Sigma} \\
 \frac{\partial \Omega(\Vparam)}{\partial \lambda} &= \delta_{\lambda \in \Lambda}^t (1-B)^{-t} \Psi (1-B)^{-1} \Lambda + \Lambda^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{\lambda \in \Lambda} \\
 \frac{\partial \Omega(\Vparam)}{\partial b} &= \Lambda^t (1-B)^{-t} \delta_{b \in B}^t (1-B)^{-t} \Psi (1-B)^{-1} \Lambda + \Lambda^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda\\
\end{align*}
#+END_EXPORT

Second order derivatives:
#+BEGIN_EXPORT LaTeX
\begin{align*}
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \alpha \partial b} &= \delta_{\alpha} (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda \\
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \alpha \partial \lambda} &= \delta_{\alpha} (1-B)^{-1} \delta_{\lambda \in \Lambda} \\
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \Gamma \partial b} &= \VX_i (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda \\
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \Gamma \partial \lambda} &= \VX_i (1-B)^{-1} \delta_{\lambda \in \Lambda} \\
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \lambda \partial b } &=  (\alpha + \VX_i \Gamma)(1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \delta_{\lambda \in \Lambda} \\
 \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial b \partial b'} &= (\alpha + \VX_i \Gamma)(1-B)^{-1}\delta_{b' \in B}(1-B)^{-1}\delta_{b \in B}(1-B)^{-1}\Lambda \\
& + (\alpha + \VX_i \Gamma)(1-B)^{-1}\delta_{b \in B}(1-B)^{-1}\delta_{b' \in B}(1-B)^{-1}\Lambda  \\
& \\
 \frac{\partial^2 \Omega(\Vparam)}{\partial \psi \partial \lambda} &=  \delta_{\lambda \in \Lambda}^t (1-B)^{-t} \delta_{\psi \in \Psi} (1-B)^{-1} \Lambda  \\
& + \Lambda^t (1-B)^{-t} \delta_{\psi \in \Psi} (1-B)^{-1} \delta_{\lambda \in \Lambda}  \\
 \frac{\partial^2 \Omega(\Vparam)}{\partial \psi \partial b} &= \Lambda^t (1-B)^{-t} \delta^t_{b \in B} (1-B)^{-t} \delta_{\psi \in \Psi} (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \delta_{\psi \in \Psi} (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda  \\
 \frac{\partial^2 \Omega(\Vparam)}{\partial \lambda \partial b} &= \delta_{\lambda \in \Lambda}^t (1-B)^{-t} \delta^t_{b \in B} (1-B)^{-t} \Psi (1-B)^{-1} \Lambda \\
& + \delta_{\lambda \in \Lambda}^t (1-B)^{-t} \Psi (1-B)^{-1} \delta^t_{b \in B} (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \delta^t_{b \in B} (1-B)^{-t} \Psi (1-B)^{-1} \delta_{\lambda \in \Lambda} \\
& + \Lambda^t (1-B)^{-t}  \Psi (1-B)^{-1} \delta^t_{b \in B} (1-B)^{-1} \delta_{\lambda \in \Lambda}  \\
 \frac{\partial^2 \Omega(\Vparam)}{\partial \lambda \partial \lambda'} &= \delta_{\lambda \in \Lambda}^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{\lambda' \in \Lambda} \\
& + \delta_{\lambda' \in \Lambda}^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{\lambda \in \Lambda}   \\
 \frac{\partial^2 \Omega(\Vparam)}{\partial b \partial b'} &= \Lambda^t (1-B)^{-t} \delta_{b' \in B}^t (1-B)^{-t} \delta_{b \in B}^t (1-B)^{-t} \Psi (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \delta_{b \in B}^t (1-B)^{-t} \delta_{b' \in B}^t (1-B)^{-t} \Psi (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \delta_{b \in B}^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{b' \in B} (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \delta_{b' \in B}^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{b' \in B} (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \Lambda \\
& + \Lambda^t (1-B)^{-t} \Psi (1-B)^{-1} \delta_{b \in B} (1-B)^{-1} \delta_{b' \in B} (1-B)^{-1} \Lambda \\
\end{align*}
#+END_EXPORT
* First derivative: score
The individual score is obtained by derivating the log-likelihood:
#+BEGIN_EXPORT LaTeX
\begin{align*}
   \Score(\param|\VY_i,\VX_i) =& \dpartial{l_i(\Vparam|\VY_i,\VX_i)}{\param}\\
 =& - \frac{1}{2} tr\left(\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param}\right) \\
 &+  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))} \\
 &+ \frac{1}{2} (\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))}
\end{align*}
#+END_EXPORT

* Second derivative: Hessian and expected information
:PROPERTIES:
:CUSTOM_ID: SM:Information
:END:
The individual Hessian is obtained by derivating twice the
log-likelihood:
#+BEGIN_EXPORT LaTeX
\begin{align*}
   \Hessian_i(\param,\param') =& -\frac{1}{2} tr\left(-\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} + \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'}\right) \\
 &+  \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \param \partial \param'} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam,\VX_i)}{\param'}} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param'} \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))}  \\
 &-  (\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))} \\
 &+ \frac{1}{2} (\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))} \\
\end{align*}
#+END_EXPORT

\clearpage

Using that \(\Vmu(\param,\VX_i)\) and \(\Omega(\Vparam)\) are deterministic quantities,
we can then take the expectation to obtain:
#+BEGIN_EXPORT LaTeX
\begin{align*}
\Esp\left[\Hessian_i(\param,\param')\right] =& -\frac{1}{2} tr\left(-\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \Vparam} + \Omega(\param)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'}\right) \\
 &+  \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial \param \partial \param'} \Omega(\Vparam)^{-1} \Ccancelto[red]{0}{\Esp\left[\trans{(\VY_i-\Vmu(\Vparam,\VX_i))}\right]} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \Ccancelto[red]{0}{\Esp\left[\trans{(\VY_i-\Vmu(\Vparam,\VX_i))}\right]} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam)}{\param'}} \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param'} \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param} \Omega(\Vparam)^{-1} \Ccancelto[red]{0}{\Esp\left[\trans{(\VY_i-\Vmu(\Vparam,\VX_i))}\right]}  \\
 &-  \Esp\left[(\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))}\right] \\
 &+ \Esp \left[\frac{1}{2} (\VY_i-\Vmu(\Vparam,\VX_i)) \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'} \Omega(\Vparam)^{-1} \trans{(\VY_i-\Vmu(\Vparam,\VX_i))}\right] \\
\end{align*}
#+END_EXPORT

The last two expectations can be re-written using that \(\Esp[\trans{x}Ax] = tr\left(A\Var[x]\right)+\trans{\Esp[x]}A\Esp[x]\):
#+BEGIN_EXPORT LaTeX
\begin{align*}
\Esp\left[\Hessian_i(\param,\param')\right] =& -\frac{1}{2} tr\left(-\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} + \Ccancel[red]{\Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'}}\right) \\
 &-  \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \param} \Omega(\Vparam)^{-1} \trans{\frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \param'}} \\
 &- tr\left(\Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} \Omega(\Vparam)^{-1} \trans{\left(\Var\left[\VY_i-\Vmu(\Vparam,\VX_i)\right]\right)} \right) \\
 &+ \Ccancel[red]{\frac{1}{2} tr\left( \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param \partial \param'} \Ccancel[blue]{\Omega(\Vparam)^{-1}} \Ccancel[blue]{\trans{\left(\Var\left[\VY_i-\Vmu(\Vparam,\VX_i)\right]\right)}} \right)} \\
\end{align*}
#+END_EXPORT
where we have used that \(\Var\left[\VY_i-\Vmu(\Vparam,\VX_i)\right] =
\Var\left[\VY_i|\VX_i\right] = \Omega(\Vparam)\). Finally we get:
#+BEGIN_EXPORT LaTeX
\begin{align*}
\Esp\left[\Hessian_i(\param,\param')\right] =& -\frac{1}{2} tr\left(\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param}\right) \\
 &-  \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam,\VX_i)}{\param'}} \\
\end{align*}
#+END_EXPORT
So we can deduce from the previous equation the expected information matrix:
#+BEGIN_EXPORT LaTeX
\begin{align*}
\Information(\param,\param') =& \frac{n}{2} tr\left(\Omega(\Vparam)^{-1} \dpartial{\Omega(\Vparam)}{\param'} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param}\right) 
 + \sum_{i=1}^n \dpartial{\Vmu(\Vparam,\VX_i)}{\param} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam,\VX_i)}{\param'}}
\end{align*}
#+END_EXPORT

* First derivatives of the information matrix
#+BEGIN_EXPORT LaTeX
\begin{align*}
\frac{\partial \Information(\param,\param')}{\partial \param''} 
=& - \frac{n}{2} tr\left(\Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param''} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param'}\right) \\
& + \frac{n}{2} tr\left( \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial\param\partial\param''} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param'}\right) \\
& - \frac{n}{2} tr\left(\Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param''} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param'}\right) \\
& + \frac{n}{2} tr\left( \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial\param} \Omega(\Vparam)^{-1} \frac{\partial^2 \Omega(\Vparam)}{\partial \param' \partial \param''}\right) \\
& + \sum_{i=1}^n \frac{\partial^2 \Vmu(\Vparam,\VX_i)}{\partial\param\partial\param''} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam,\VX_i)}{\param'}} \\
& + \sum_{i=1}^n \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \param} \Omega(\Vparam)^{-1} \trans{\ddpartial{\Vmu(\Vparam,\VX_i)}{\Vparam'}{\param''}} \\
& - \sum_{i=1}^n \frac{\partial \Vmu(\Vparam,\VX_i)}{\partial \param} \Omega(\Vparam)^{-1} \frac{\partial \Omega(\Vparam)}{\partial \param''} \Omega(\Vparam)^{-1} \trans{\dpartial{\Vmu(\Vparam,\VX_i)}{\param'}} \\
\end{align*}
#+END_EXPORT


* Config                                                           :noexport:

#+LANGUAGE:  en
#+LaTeX_CLASS: org-article
#+OPTIONS: author:t date:nil todo:t title:t toc:nil
#+LaTeX_class_options: [table] 
#+STARTUP: content

** R code 
#+PROPERTY: session *R*

** Margins and space between lines
# ## set margin
#+LATEX_HEADER: \geometry{innermargin=1.5in,outermargin=1.25in,vmargin=3cm}
# ## set space between lines
#+LATEX_HEADER: \linespread{1.4}

** Figures
#+LATEX_HEADER: \usepackage{epstopdf} % to be able to convert .eps to .pdf image files
#+LATEX_HEADER: \renewcommand{\thefigure}{S\arabic{figure}}
#+LATEX_HEADER: \renewcommand{\thetable}{S\arabic{table}}
#+LATEX_HEADER: \renewcommand{\theequation}{S\arabic{equation}}

# # for figure S1
#+LaTeX_HEADER: \usepackage{caption}
#+LaTeX_HEADER: \usepackage[labelformat=simple]{subcaption}
#+LaTeX_HEADER: \renewcommand{\thesubfigure}{Study \Alph{subfigure}}

** Tables 
#+LaTeX_HEADER: \usepackage{booktabs}

** Algorithm
#+LATEX_HEADER: \usepackage{algorithm2e}
#+LaTeX_HEADER: \usepackage{amsthm}
 
** Math - shortcut
#+LATEX_HEADER: \usepackage{amsthm,dsfont,amsmath}
#+LaTeX_HEADER: \newtheorem{lemma}{Lemma}

#+LaTeX_HEADER:\newcommand{\Vn}{\mathbf{n}}
#+LaTeX_HEADER:\newcommand{\X}{X}
#+LaTeX_HEADER:\newcommand{\VX}{\boldsymbol{X}}
#+LaTeX_HEADER:\newcommand{\Y}{Y}
#+LaTeX_HEADER:\newcommand{\VY}{\boldsymbol{Y}}
#+LaTeX_HEADER:\newcommand{\Vy}{\boldsymbol{y}}
#+LaTeX_HEADER:\newcommand{\VZ}{\boldsymbol{Z}}
#+LaTeX_HEADER:\newcommand{\Veta}{\boldsymbol{\eta}}
#+LaTeX_HEADER:\newcommand{\Vvarepsilon}{\boldsymbol{\varepsilon}}

#+LaTeX_HEADER:\newcommand{\set}{\mathcal{S}}
#+LaTeX_HEADER:\newcommand{\Vmu}{\boldsymbol{\mu}}
#+LaTeX_HEADER:\newcommand{\Vxi}{\boldsymbol{\xi}}

#+LaTeX_HEADER:\newcommand{\param}{\theta}
#+LaTeX_HEADER:\newcommand{\paramHat}{\hat{\param}}
#+LaTeX_HEADER:\newcommand{\Vparam}{\boldsymbol{\param}}
#+LaTeX_HEADER:\newcommand{\VparamHat}{\boldsymbol{\paramHat}}

#+LATEX_HEADER: \newcommand\Hessian{\mathcal{H}}
#+LATEX_HEADER: \newcommand\Likelihood{\mathcal{L}}
#+LATEX_HEADER: \newcommand\Information{\mathcal{I}}
#+LATEX_HEADER: \newcommand\Score{\mathcal{U}}
#+LATEX_HEADER: \newcommand\Hypothesis{\mathcal{H}}

#+LATEX_HEADER: \newcommand\Real{\mathbb{R}}
#+LaTeX_HEADER: \newcommand\half{\frac{1}{2}}

** Math - operator

#+LATEX_HEADER: \newcommand\Ind[1]{\mathds{1}_{#1}}
#+LATEX_HEADER: \newcommand\dpartial[2]{\frac{\partial #1}{\partial #2}}
#+LATEX_HEADER: \newcommand\ddpartial[3]{\frac{\partial^2 #1}{\partial #2 \partial #3}}

#+LATEX_HEADER: \newcommand\Esp{\mathbb{E}}
#+LATEX_HEADER: \newcommand\Var{\mathbb{V}ar}
#+LATEX_HEADER: \newcommand\Cov{\mathbb{C}ov}
#+LATEX_HEADER: \newcommand\Gaus{\mathcal{N}}

#+LATEX_HEADER: \newcommand\trans[1]{{#1}^\intercal}%\newcommand\trans[1]{{\vphantom{#1}}^\top{#1}}

#+LATEX_HEADER: \newcommand{\independent}{\mathrel{\text{\scalebox{1.5}{$\perp\mkern-10mu\perp$}}}}

** Math - cancel
#+LaTeX_HEADER: \RequirePackage[makeroom]{cancel} 
#+LaTeX_HEADER: \newcommand\Ccancelto[3][black]{\renewcommand\CancelColor{\color{#1}}\cancelto{#2}{#3}}
#+LaTeX_HEADER: \newcommand\Ccancel[2][black]{\renewcommand\CancelColor{\color{#1}}\cancel{#2}}

** Local Words
#  LocalWords:  REML JRSS Kenward bootstrapLavaan boldsymbol Veta VX
#  LocalWords:  Vvarepsilon sim Gaus varepsilon eq notag frac mathcal
#  LocalWords:  df qquad ldots nabla dpartial nablaFtheta ddpartial
#  LocalWords:  biasVcov OmegaOmegahat infty eqref biasOmega VY BDNF
#  LocalWords:  correctedN dScoredY RestyleAlgo boxruled textbf ATTR
#  LocalWords:  leftarrow intercal widehat HTTLPR begingroup endgroup
#  LocalWords:  renewcommand textwidth subfigure includegraphics
#  LocalWords:  linewidth GraphSimul graphFactorModel orgmode
