080. ANOVA

Unified Approach to Balanced ANOVA Models

We can develop a unified approach to obtaining orthogonal projection operatores in arbitrary balanced $k$-way ANOVA models by exploting the structure of design matrix. The structure of the design matrix can be easily examined using Kronecker products. Therefore, before we proceed further, we need to establish some more properties of Kronecker products.

Kronecker Product $:=A\otimes B=(a_{ij}B)$.

Consider the balanced two-way ANOVA model with interaction. This model is given by

$$Y_{ijk}=\mu +{\alpha }_i+{\eta }_j+{\gamma }_{ij}+{ϵ}_{ijk}$$

where $i=1,\cdots ,a$, $j=1,\cdots ,b$, $k=1,\cdots ,N$, and $n=abN$.

We want to write

$$\mathcal{C}(M)=\mathcal{C}(M_{\mu })+\mathcal{C}(M_{\alpha })+\mathcal{C}(M_{\eta })+\mathcal{C}(M_{\gamma })$$

and be able to compute the orthogonal projection operators in an easy and unified way.

We can represent each subspace making up $\mathcal{C}(M)$ in terms of Kronecker produdcts. Once we do this, we can easily obtain the orthogonal projection operator for that space.

※ Notation: let $s$ be an arbitrary index. Define $J_s$ as the $s\times 1$ vector of ones, $P_s=\frac{1}{s}{J}_s{J}_s^{\prime }$ and $Q_s=I_s-P_s$, where $I_s$ is the $s\times s$ identity matrix.

Thus, $P_s$ is the orthogonal projection operator onto $\mathcal{C}(J_s)$ and $Q_s$ is the orthogonal projection operator onto $\mathcal{C}(J_s{)}^{\perp }$

※ Facts:

1. recall that the OPO onto $\mathcal{C}(A)$ is always given by by $A(A^{\prime }A{)}^{-}{A}^{\prime }$.
2. if $M$ is an OPO, then $M^{-}=M$.

---

Kronecker Product forms for the OPO

1. Computing $M_{\mu }$.

We can write $J_n=J_{\text{a}}\otimes J_b\otimes J_N$, so that $M_{\mu }$ is the OPO onto $\mathcal{C}(J_{\text{a}}\otimes J_b\otimes J_N)$. Thus by Fact 1 above, we have

$$\begin{array}{rlrlrlrlrlr}& & & M_{\mu }& & & & & & & \\ & =& & (J_a\otimes J_b\otimes J_N)& & ((J_a^{\prime }\otimes J_b^{\prime }\otimes J_N^{\prime })(J_a\otimes J_b\otimes J_N){)}^{-}& & (J_a^{\prime }\otimes J_b^{\prime }\otimes J_N^{\prime })& & & \\ & =& & (J_a\otimes J_b\otimes J_N)& & (J_a^{\prime }{J}_a\otimes J_b^{\prime }{J}_b\otimes J_N^{\prime }{J}_N{)}^{-}& & (J_a^{\prime }\otimes J_b^{\prime }\otimes J_N^{\prime })& & & \\ & =& & (J_a\otimes J_b\otimes J_N)& & (abN{)}^{-}& & (J_a^{\prime }\otimes J_b^{\prime }\otimes J_N^{\prime })& =& & \frac{1}{a}{J}_a{J}_a^{\prime }+\frac{1}{b}{J}_b{J}_b^{\prime }+\frac{1}{N}{J}_N{J}_N^{\prime }\\ & =& & P_a\otimes P_b\otimes P_N& & & & & & & \end{array}$$

Using the properties of Kronecker products, it can be easily shown that $M=I_a\otimes I_b\otimes P_N$.

the error space is $\mathcal{C}(I-M)$ and

$$\begin{array}{rl}I-M& =I_{abN}-M\\ & =(I_a\otimes I_b\otimes I_N)-(I_a\otimes I_b\otimes P_N)\\ & =(I_a\otimes )\otimes (I_N-P_N)\\ & =I_a\otimes I_b\otimes Q_N\end{array}$$

observe that

$$\begin{array}{rl}M+I-M& =(I_a\otimes I_b\otimes P_N)+(I_a\otimes I_b\otimes Q_N)\\ & =(I_a\otimes I_b)\otimes (P_N+Q_N)\\ & =(I_a\otimes I_b)\otimes (I_N)\\ & =I_a\otimes I_b\otimes I_N\\ & =I_n\end{array}$$

We can summarize the subspace and the OPO for the two-way ANOVA model as follows.

• Excercise

Consider the three-way ANOVA model

1. write out the subspaces and all OPO corresponding to each term in the ANOVA model completlely in terms of Kronecker.

2. Find the simplest expression for $M_{\mu }+M_{\alpha }+M_{\eta }$.