040. 1Way-ANOVA

One-Way ANOVA

One-Way ANOVA

General form of One-Way ANOVA model is

$$y_{ij}=\mu +{\alpha }_i+{ϵ}_{ij},i=1,\cdots ,aj=1,\cdots ,N_i$$ $$\begin{gathered} n=\sum _{i=1}^a{N}_i \\ E({ϵ}_{ij})=0,Var({ϵ}_{ij})={\sigma }^2,Cov({ϵ}_{ij},{ϵ}_{ab})=0 \end{gathered}$$

• i-th treatment (group) effect $a_i$
  • Balanced model is $\forall i:N_i=b$
  • Unbalanced model is $\forall i:N_i$‘s are different

---

More About Models

• Example 4.1.1:

$a=3,N_1=5,N_2=3,N_3=3$,

$$Y=X\beta +ϵ=\left(\begin{array}{cccc}{J}_5& J_5& 0& 0\\ J_3& 0& J_3& 0\\ J_3& 0& 0& J_3\end{array}\right)\left(\begin{array}{c}\mu \\ {\alpha }_1\\ {\alpha }_2\\ {\alpha }_3\end{array}\right)+\left(\begin{array}{c}{ϵ}_{11}\\ {ϵ}_{12}\\ ⋮\\ {ϵ}_{33}\end{array}\right)$$

let $N_1=N_2=N_3=5$. then

$$X=\left(\begin{array}{cc}{J}_3\otimes J_5& I_3\otimes J_5\end{array}\right)$$

In general, balanced design such as $i=1,\cdots ,aj=1,\cdots ,b$:

$$X=\left(\begin{array}{cc}{J}_a\otimes J_b& I_a\otimes J_b\end{array}\right)$$

• Notation: $J_r^c\equiv J_r{J}_c^{\prime }=J_r\otimes J^c$ is a $r\times c$ matrix of $1$‘s.

Let $Z$ be the model matrix for the alternative one-way analysis of variance model

$$y_{ij}={\mu }_i+{ϵ}_{ij}i=1,\cdots ,ak=1,\cdots ,N_i$$

then, letting $X_i{X}_j={\delta }_{ij}$ with 1 for $i=j$ and 0 for $i\ne j$,

$$\begin{array}{rlrl}X& =\left[\begin{array}{cc}J& Z\end{array}\right]& & =\left[\begin{array}{cc}J& (X_1,\cdots ,X_a)\end{array}\right]\\ ⟹\mathcal{C}(X)& =\mathcal{C}(Z)& & \\ Z^{\prime }Z& =diag(N_1,N_2,\cdots ,N_a)& & \\ Z(Z^{\prime }Z{)}^{-1}{Z}^{\prime }& =Blkdiag\left[N_i^{-1}{J}_{N_i}^{N_i}\right]& & \\ M& =X(X^{\prime }X{)}^{-1}{X}^{\prime }& & \\ M_{\alpha }& =Z_{\ast }(Z_{\ast }^{\prime }{Z}_{\ast }{)}^{-1}{Z}_{\ast }^{\prime }& & =M-M_J=M-\frac{1}{n}{J}_n^n\\ Z_{\ast }& =(I-M_j)Z& & \\ M& =M_j+M_{\alpha }& & \end{array}$$

---

Estimating and Testing Contrasts

A contrast in the one-way ANOVA

$${\lambda }^{\prime }\beta =\sum _{i=1}^a{\lambda }_i{\alpha }_{i}with{\lambda }^{\prime }{J}_{a+1}=\sum _{i=1}^a{\lambda }_i=0$$

For estimable ${\lambda }^{\prime }\beta$, find $\rho$ so that ${\rho }^{\prime }X={\lambda }^{\prime }$, ${\rho }^{\prime }=\left(\begin{array}{c}\frac{J_{N_i}^{\prime }{\lambda }_i}{N_i}\end{array}\right)$.

• Proposition 4.2.1.

${\lambda }^{\prime }\alpha ={\rho }^{\prime }X\beta$ is a contrast $⟺$ ${\rho }^{\prime }J=0$.

• Proposition 4.2.2.

${\lambda }^{\prime }\alpha ={\rho }^{\prime }X\beta$ is a contrast $⟺$ $M_{\rho }\in \mathcal{C}(M_{\alpha })$.

since ${\sum }_{i=1}^a{\lambda }_i=0$,

$$\sum _{i=1}^a{\lambda }_i{\hat{\alpha }}_i=\sum _{i=1}^a{\lambda }_i\{\hat{\mu }+{\hat{\alpha }}_i\}=\sum _{i=1}^a{\lambda }_i{\bar{y}}_{i+}$$

because $\mu +{\alpha }_i$ is estimable, and its unique LSE is ${\bar{y}}_{i+}$.

At significance level $\alpha$, $H_0:{\lambda }^{\prime }\alpha =0$ is rejected if

\begin{alignat}{2} &F &&= \dfrac { \dfrac{ \Big( \sum_{i=1}^a \lambda_i \bar y_{i+} \Big) ^2} {\dfrac{\sum_{i=1}^a \lambda_i^2}{N_i}} } {MSE} &&> F \Big(1-\alpha, \; \; 1, \; \; dfE \Big) \\ \\ \\ \iff \; \; \; \; \; & t \ &&= \dfrac {\Bigg \vert \sum_{i=1}^a \lambda_i \bar y_{i+} \Bigg \vert} {\sqrt{MSE \left( \sum_{i=1}^a\dfrac{\lambda_i^2}{N_i}\right) }} &&> t \left( 1-\dfrac{\alpha}{2}, \; \; dfE \right) \end{alignat}

---

Cochran’s Theorem

let $A_1,\cdots ,A_m$ be $n\times n$ symmetric Matrices, and $A={\sum }_{j=1}^m{A}_j$ with $rank(A_j)=n_j$. consider the following four statements:

1. $A_j$ is an orthogonal projection for all $j$.
2. $A$ is an orthogonal projection (possibly $A=I$).
3. $A_j{A}_k=0$ for all $j\ne k$.
4. ${\sum }_{j=1}^m{n}_j=n$.

If any two of these conditions hold, then all four hold.

• Note: Cochran’s theorem is a standard result that is the basis of the ANalysis Of VAriance. If we can write the total sum of squares as a sum of sum of squares components, and if the degree of freedom add up, then the $A_j$ must be projections, they are orthogonal to each other, and they jointly span ${\mathbb{R}}^n$.