020. Introduction

Introduction

What

for linear model $Y=X\beta +ϵ$

$$\begin{gathered} Y_{n\times 1}=\left(\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right){\beta }_{(p+1)\times 1}=\left(\begin{array}{c}{\beta }_0\\ ⋮\\ {\beta }_p\end{array}\right){ϵ}_{n\times 1}=\left(\begin{array}{c}{ϵ}_1\\ ⋮\\ {ϵ}_n\end{array}\right) \\ {X}_{n\times (p+1)}=\left(\begin{array}{cccc}1& X_{11}& \cdots & X_{1p}\\ 1& X_{21}& \cdots & X_{2p}\\ ⋮& & \ddots & ⋮\\ 1& X_{n1}& \cdots & X_{np}\end{array}\right) \end{gathered}$$

• linear regression

$$\begin{array}{rlrl}{y}_i& ={\beta }_0+{\beta }_1{x}_i& & +{ϵ}_i\\ y_i& ={\beta }_0+\sum _{j=1}^p{\beta }_j{x}_{ij}& & +{ϵ}_i\end{array}\text{\hspace{1em}(Simple)(Multiple)}$$

• ANOVA

$$\begin{array}{rlrl}{y}_{ij}& =\mu +{\alpha }_i& & +{ϵ}_{ij}\\ y_{ij}& =\mu +{\alpha }_i+{\beta }_j+(\alpha \beta {)}_{ij}& & +{ϵ}_{ij}\end{array}\text{\hspace{1em}(One-Way)(Two-Way with interaction)}$$

---

Random Vectors and Matrices

let rv $Y={\left(\begin{array}{ccc}{y}_1,& \cdots & ,y_n\end{array}\right)}^{\prime }$ with $E(y_i)={\mu }_i,Var(y_i)={\sigma }_{ii}(={\sigma }_i^2),Cov(y_i,y_j)={\sigma }_{ij}$.

• define the statistics of $Y$

$$\begin{array}{rlrl}E(Y)& ={\left(\begin{array}{ccc}E(y_1),& \cdots & E(y_n)\end{array}\right)}^{\prime }={\left(\begin{array}{ccc}{\mu }_1,& \cdots & {\mu }_n\end{array}\right)}^{\prime }& & =\mu \\ Cov(Y)& =E\left[(Y-\mu )(Y-\mu {)}^{\prime }\right]& & =({\sigma }_{ij})\end{array}\text{\hspace{1em}(Expected Value of Y elementwise as )(Covariance Matrix)}$$

• Note:

$$\begin{array}{rl}E(AY+b)& =A\mu +b\\ Cov(AY+b)& =A\ast Cov(Y)\ast A^{\prime }\end{array}$$

• Prove or disprove that Cov(Y) is nonnegative definite. how?

---

Covariance of $W_{r\times 1},Y_{s\times 1}$ with $E(W)=\gamma ,E(Y)=\mu$:

$$\begin{array}{rlrl}Cov(W,Y)& =E{\left[(W-\gamma )(Y-\mu {)}^{\prime }\right]}_{r\times s}& & \\ Cov(AW+a,NY+b)& =A\ast Cov(W,Y)\ast B^{\prime }& & \\ Cov(AW+NY)& =A\ast Cov(W)\ast A^{\prime }+N\ast Cov(Y)\ast B^{\prime }& & \\ & +A\ast Cov(W,Y)\ast B^{\prime }+B\ast Cov(W)\ast A^{\prime }& & \end{array}\text{\hspace{1em}(why?)}$$

---

Multivariate Normal Distributions

$$Z=(z_1,\cdots ,z_n{)}^{\prime }\sim N_n(0,I_n),z_1,\cdots ,z_n\stackrel{iid}{\sim }N(0,1)$$

which means $E(Z)=0,Cov(Z)=I_n$.

$$A_{r\times n},b\in {\mathbb{R}}^r$$

Y has an r-dimensional MVN distribution

Definition 1.2.1. Let A be r  n and b 2 Rr . Then Y has an r-dimensional multivariate normal distribution : Y = AZ + b  Nr (b;AAT ): Theorem 1.2.2. Let Y  N(;V) and W  N(;V). Then Y and W have the same distribution (Proof: p.5)

The density of nonsingular $Y\sim N(\mu ,V)$ is given by

$$f(y)=(2\pi {)}^{-\frac{n}{2}}{\left[\det (V)\right]}^{-\frac{1}{2}}\exp \left[-\frac{1}{2}(y-\mu {)}^{\prime }{V}^{-1}(y-\mu )\right]$$

Theorem 1.2.3. Let Y  N(;V) and Y =

Y1 Y2 ! . Then Cov(Y1;Y2) = 0 if and only if Y1 Y2 Corollary 1.2.4. Let Y  N(; 2I) and ABT = 0. Then AY BY

Definition 1.3.1. Quadratic Form of Y: for n  n; A YTAY = X ij aijyiyj Theorem

---

Distributions of Quadratic Forms

$E(Y)=\mu ,Cov(Y)=V$. then $E(Y^{\prime }AY)=tr(AV)+{\mu }^{\prime }A\mu$. prf)

let’s consider $Z\sim N_n(\mu ,I_n)$. then Z'Z \sim \chi^2 \left(n, \; \dfrac{\mu' \mu}{2} \right) \tag{second one is non-centrality parameter}

Let $Y\sim N(\mu ,I)$ and any orthogonal projection Matrix $M$. then $Y^{\prime }MY\sim {\chi }^2\left(r(M),\frac{{\mu }^{\prime }M\mu }{2}\right)$

Let $Y\sim N(\mu ,{\sigma }^{2}I)$ and any orthogonal projection Matrix $M$. then $Y^{\prime }MY\sim {\chi }^2\left(r(M),\frac{{\mu }^{\prime }M\mu }{2{\sigma }^2}\right)$

Let $Y\sim N(\mu ,M)$ with $\mu \in \mathcal{C}(M)$ and $M$ be an orthogonal projection Matrix. then $Y^{\prime }Y\sim {\chi }^2\left(r(M),\frac{{\mu }^{\prime }M\mu }{2{\sigma }^2}\right)$ let $E(Y)=\mu, ; Cov(Y)=V$.then$Pr \left[ (Y-\mu) \in \mathcal{C}(V) \right]=1$$

• Exercise 1.6.

Let $Y$ be a vector with $E(Y)=0$ and $Cov(Y)=0$. Then $Pr(Y=0)=1$ Let $Y\sim N(\mu ,V)$. then $Y^{\prime }AY\sim {\chi }^2\left(tr(AV),\frac{{\mu }^{\prime }A\mu }{2}\right)$, provided that

1. $VAVAV=VAV$.
2. ${\mu }^{\prime }AVA\mu ={\mu }^{\prime }a\mu$.
3. $VAVA\mu =VA\mu$

prf)

• Exercise 1.7.

1. Show that if $V$ is nonsingular, then the three conditions in Theorem 1.3.6 reduce to $AVA=A$.
2. Show that $Y^{\prime }{V}^{-}Y$ has a chi-squared distribution with $r(V)$ degrees of freedom when $\mu \in \mathcal{C}(V)$.

let $Y\sim N(\mu ,{\sigma }^{2}I)$ and $BA=0$. then, for $A=A^{\prime }$,

1. $Y^{\prime }AY\perp BY$
2. $Y^{\prime }AY\perp Y^{\prime }BY$ for $B=B^{\prime }$

let $Y\sim N(\mu ,V)$ and $A\ge 0,B\ge 0$, and $VAVBV=0$. then $Y^{\prime }AY\perp Y^{\prime }BY$.

let $Y\sim N(\mu ,V)$. provided that

1. $VAVBV=0$.
2. $VAVB\mu =0$.
3. $VBVA\mu =0$.
4. ${\mu }^{\prime }ABV\mu =0$.

and also conditions of above thm,

1. $VAVAV=VAV$.
2. ${\mu }^{\prime }AVA\mu ={\mu }^{\prime }a\mu$.
3. $VAVA\mu =VA\mu$

prf)

hold for both $Y^{\prime }AY$ and $Y^{\prime }BY$, then $Y^{\prime }AY\perp Y^{\prime }BY$.