Testing

More About Models: Two approaches for linear model

Y Y = E (Y) = μ = E (Y) = Xβ + Y - E (Y) + ϵ + Y - E (Y) + ϵ (Parameter-free approach) (Parameter approach)

\begin{alignat}{2} E(\epsilon) &= 0, \; \; \; && Cov(\epsilon) &&= \sigma^2 I \tag{Ordinary Least Square, OLS} \\ E(\epsilon) &= 0, && Cov(\epsilon) &&= \sigma^2 \Sigma \tag{Generalized Least Square, GLS} \end{alignat}

Consider

Y = Xβ + ϵ, E (ϵ) = 0, C o v (ϵ) = σ^{2} I

	$C (X)$	$C (X)^{⊥}$
itslef	Estimation Space	Error Space
orthogonal projection onto	$M = X (X^{'} X)^{-} X^{'}$	$I - M = I - X (X^{'} X)^{-} X^{'}$
	$E (Y) = Xβ \in C (X)$	$E (ϵ) \in C (X)^{⊥}$
	$C o v (Y) = σ^{2} I$	$C o v (ϵ) = σ^{2} I$

One-Way ANOVA

y_{ij} \overset{μ}{ˉ} μ_{1} - μ_{2} = μ_{i} = E (y_{ij}) = μ + α_{i} = μ + \overset{α}{ˉ}_{+} = α_{1} - α_{2} + ϵ_{ij} + ϵ_{ij} + ϵ_{ij}

the parameters in the two models are different, but they are related.

Simple Linear Regression

y_{i} = β_{0} + β_{1} x_{i} = E (y_{i}) = γ_{0} + γ_{1} (x_{i} - \overset{x}{ˉ}) + ϵ_{i} + ϵ_{i} + ϵ_{i}

C (X_{1}) = C (X_{2}) ⟹ X_{1} X_{1} β_{1} = X_{2} T = X_{2} T β_{1} = X_{2} β_{2} = X_{2} (T β_{1} + ν), \forall ν \in C (X_{2}^{'})^{⊥}

※ Note: A unique parameterization for $X_{j}, j = 1, 2$ occurs $⟺$ $X_{j}^{'} X_{j}$ is nonsingular.

Exercise: Show that a unique parameterization for $X_{j}, j = 1, 2$ means $C (X_{2}^{'})^{⊥} = {0}$ .

Testing Models

Consider

Y = Xβ + ϵ, ϵ \sim N (0, I_{n})

let’s partition $X$ into $X = (X_{0}, X_{1}) : C (X_{0}) \subset C (X)$

Y Y = X_{0} β_{0} + X_{1} β_{1} = X_{0} γ + ϵ + ϵ (Full Model, FM) (Reduced Model, RM)

이때 Hypothesis testing procedure can be described as $H_{0} :$ Reduced Model, $H_{1} :$ Full Model. (Example 3.2.0: pp. 52–54).

Let $M$ and $M_{0}$ be the orthogonal projection onto $C (X)$ and $C (X_{0})$ respectively.

Note that with $C (X_{0}) \subset C (X)$ , $M - M_{0}$ is the orthogonal projection onto the orthogonal complement of $C (X_{0})$ with respect to $C (X)$ , that is,

C (X_{0})_{C (X)}^{⊥} \overset{μ}{^} \overset{μ}{^}_{0} = C (M - M_{0}) = C (M \cap M_{0}^{⊥}) = \hat{E} (Y) = M Y = \hat{E} (Y) = M_{0} Y (under FM) (under RM)

If RM is true, then $M Y - M_{0} Y = (M - M_{0}) Y$ should be reasonably small. Note that $E (M - M_{0}) Y = 0$ .

The decision about whether RM is appropriate hinges on deciding whether the vector $(M - M_{0}) Y$ is large.

The size of $(M - M_{0}) Y$ ‘s obvious measure is $[(M - M_{0}) Y]^{'} [(M - M_{0}) Y] = Y^{'} (M - M_{0}) Y$ .

The size of $(M - M_{0}) Y$ ‘s reasonable measure is given by $\frac{Y ^{'} ( M - M _{0} ) Y}{r ( M - M _{0} )}$ .

※ Note that $E (\frac{Y ^{'} ( M - M _{0} ) Y}{r ( M - M _{0} )}) = σ^{2} + \frac{β ^{'} X ^{'} ( M - M _{0} ) Xβ}{r ( M - M _{0} )}$ .
Theorem 3.2.1.

Consider

Y = Xβ + ϵ, ϵ \sim N (0, I_{n}), C (X_{0}) \subset C (X) Y Y = X_{0} β_{0} + X_{1} β_{1} = X_{0} γ + ϵ + ϵ (Full Model, FM) (Reduced Model, RM)

\begin{alignat}{2} \dfrac {\dfrac{Y'(M-M_0)Y}{r(M-M_0)}} {\dfrac{Y'(I-M)Y}{r(I-M)}} &= \dfrac {\dfrac{Y'(M-M_0)Y}{df_1}} {\dfrac{Y'(I-M)Y}{df_2}} &&\sim F \Bigg( df_1 , df_2, \dfrac{\beta' X' (M-M_0)X \beta }{2 \sigma^2} && \Bigg) \tag{Under the FM} \\ \\\ \\\ \dfrac {\dfrac{Y'(M-M_0)Y}{r(M-M_0)}} {\dfrac{Y'(I-M)Y}{r(I-M)}} &= \dfrac {\dfrac{Y'(M-M_0)Y}{df_1}} {\dfrac{Y'(I-M)Y}{df_2}} &&\sim F \big( df_1 , df_2, 0 && \big) \tag{Under the RM} \end{alignat}

Note: Example 3.2.2.; pp. 58–59

M - M_{0} Y^{'} (M - M_{0}) Y = (I - M_{0}) = Y^{'} (I - M_{0}) Y = SS E_{RM} - (I - M) - Y^{'} (I - M) Y - SS E_{FM}

A Generalized Test Procedure

Assume that $Y = Xβ + ϵ$ is correct. Want to test the adequacy of a model $Y = X_{0} γ + X b + ϵ$ , where $C (X_{0}) \subset C (X)$ and some known vector $X b =$ offset.

Example 3.2.3.; Multiple Regression

Y = β_{0} J + β_{1} X_{1} + β_{2} X_{2} + β_{3} X_{3} + ϵ

want to test $H_{0} : β_{2} = β_{3} + 5, b e t a_{1} = 0, \dots$ .

\begin{alignat}{2} Y &= X \beta && &&+ \epsilon \tag{FM} \\ Y^\ast &\equiv Y && - X b && \\ &=X \beta && - Xb &&+ \epsilon \\ &=X (\beta && - b) &&+ \epsilon \\ &=X \beta^\ast && &&+ \epsilon \tag{FM} \\ \\\ \\\ Y &= X_0 \gamma && + Xb &&+ \epsilon \tag{RM} \\ Y^\ast &\equiv Y && && && - X b \\ &=X \gamma && &&+ \epsilon \tag{RM} \end{alignat}

In addition, when $Y^{*} = Y_{*}$ ,

\frac{\frac{Y _{*}^{'} ( M - M _{0} ) Y _{*}}{r ( M - M _{0} )}}{\frac{Y _{*}^{'} ( I - M ) Y _{*}}{r ( I - M )}} δ^{2} \sim F (r (M - M_{0}), r (I - M), δ^{2}) = \frac{1}{2 σ ^{2}} (β^{*}^{'} X^{'} (M - M_{0}) X β^{*}) (non-centrality parameter)

00 Xβ = β_{*}^{'} X^{'} ⇕ = ⇕ = M_{0} (X (M - M_{0}) X β_{*} (M - M_{0}) X β_{*} β - X b) + X b (3)

(3) will hold if

γ = (X_{0}^{'} X_{0})^{-} X_{0} (Xβ - X b) = (X_{0}^{'} X_{0})^{-} X_{0} X β_{*}

Furthermore,

\begin{alignat}{2} Y_\ast ' (M-M_0)Y_\ast &= Y_\ast ' (I-M_0)Y_\ast &&- &&Y_\ast ' (I-M)Y_\ast \; \; \; \; \;\text{ , and } \\ Y ' (I-M)Y &= && && Y_\ast ' (I-M)Y_\ast \end{alignat}

Testing Linear Parametric Functions

$H_0: Y= X \beta + \epsilon, \; \; \; \; \; \Lambda' \beta=0 \tag{1}$

Λ^{'} β = 0 ⟺ β ⟺ β ⊥ C (Λ) ⟺ β ⊥ C (Γ) ⟺ β ⊥ C (U) ⟺ β = U_{γ} \in N (Λ^{'}) = C (X)^{⊥} if \existsΓ s . t . C (Γ) = C (Λ) if \exists U s . t . C (U) = C (Λ)^{⊥} \exists γ (2)

Thus, letting $X_{0} = X U$ , (in general, $C (X_{0}) \subset C (X)$ ), then

Y = Xβ = X U γ = X_{0} γ + ϵ + ϵ + ϵ (3)

Suppose $C (X_{0}) = C (X)$ . Then there is nothing to test and $Λ^{'} β = 0$ involves only arbitrary side conditions that do not affect the model. (EXAMPLE 3.3.1. pp. 62–64)

Λ^{'} β is estimable ⟺ \exists P : Λ = X^{'} P

C (MP) \equiv C (M - M_{0}) = C (X - X_{0}) = C (X) \cap C (X_{0})^{⊥} = C (X_{0})_{C (X)}^{⊥}

thus, its distribution for testing $H_{0} : Λ^{'} β = 0$ is given by

\frac{\frac{Y ^{'} ( M _{MP} ) Y}{r ( M _{MP} )}}{\frac{Y ^{'} ( I - M ) Y}{r ( I - M )}} δ^{2} \sim F (r (M_{MP}), r (I - M), δ^{2}) = β^{'} X^{'} M_{MP} Xβ (5) (non-centrality parameter)

Proposition 3.3.2

C (M - M_{0}) = C (X_{0})_{C (X)}^{⊥} = C (X U)_{C (X)}^{⊥} = C (MP)

H_{0} : Y = Xβ + ϵ, Λ^{'} β = 0 ⇕ H_{0} : Y = Xβ + ϵ, P^{'} Xβ = 0 ⇕ H_{0} : Y = Xβ + ϵ, P^{'} MXβ = 0 (∵ MX = X) ⇕ E (Y) \in C (X), E (Y) ⊥ C (MP) ⇕ E (Y) \in C (X) \cap C (MP)^{⊥}, C (X_{0}) = C (X) \cap C (MP)^{⊥} = C (MP)_{C (X)}^{⊥} ⟹ C (X_{0})_{C (X)}^{⊥} = C (MP) ⇕ X_{0} = (I - M_{MP}) X

\begin{align} \mathcal{C} \Big[ (I-M_{MP})X \ \Big] &= \mathcal{C} (X) \; \cap \; \mathcal{C} (MP)^\perp \\ &= \mathcal{C} (X) \; \cap \; \mathcal{C} (P)^\perp \tag{EXAMPLE 3.3.4.: pp.66–67} \end{align} $ let $\Lambda ' \beta$ is estimable, i.e., $\Lambda = X'P$. then $\mathcal{C}(\Lambda) = \mathcal{C}(X'P) =\mathcal{C}(MP)$, and $X \hat \beta = MY$, and $\Lambda ' \hat \beta = P' X \hat \beta = P' M Y$. then

Y^{'} M_{MP} Y = Y^{'} M = \hat{β}^{'} Λ = \hat{β}^{'} Λ (P^{'} MP)^{-} [P^{'} X (X^{'} X)^{-} X^{'} P]^{-} [Λ^{'} (X^{'} X)^{-} Λ]^{-} MP Y Λ^{'} \hat{β} Λ^{'} \hat{β}

이윗부분 전혀모르겠음

thus,

(5) = \frac{\frac{β ^ ^{'} Λ [ Λ ^{'} ( X ^{'} X ) ^{-} Λ ] ^{-} Λ ^{'} β ^}{r ( Λ )}}{MSE} δ^{2} C o v (Λ^{'} \hat{β}) \sim F (r (MP), r (I - M), δ^{2}) = \frac{β ^ ^{'} Λ [ Λ ^{'} ( X ^{'} X ) ^{-} Λ ] ^{-} Λ ^{'} β ^}{2 σ ^{2}} = σ^{2} Λ^{'} (X^{'} X)^{-} Λ

For $H_{0} : λ^{'} β = 0, λ \in R^{p}$ ,

Y^{'} M_{MP} Y = \hat{β}^{'} λ [λ^{'} (X^{'} X)^{-} λ]^{-} λ^{'} \hat{β} = \frac{( λ ^{'} β ^ ) ^{2}}{λ ^{'} ( X ^{'} X ) ^{-} λ}

and, under $H_{0} : λ^{'} β = 0$ ,

F = (5) = \frac{( λ ^{'} β ^ ) ^{2}}{MSE [ λ ^{'} ( X ^{'} X ) ^{-} λ ]} \sim F (1, r (I - M))

Definition 3.3.5.

The condition $E (Y) ⊥ C (MP)$ is called the constraint by $Λ^{'} β = 0$ where $Λ = X^{'} P$ . in other words, $C (MP)$ is the constraint by $Λ^{'} β = 0$ .

Do Exercise 3.5:

Show that a necessary and sufficient condition for $ρ_{1}^{'} Xβ = 0$ and $ρ_{2}^{'} Xβ = 0$ to determine the orthogonal constraints on the model is that $ρ_{1}^{'} X ρ_{2} = 0$

Theoretical Complements

Consider testing $Λ^{'} β = 0$ when $Λ^{'} β$ is NOT estimable.

let $Λ_{0}^{'} β$ be estimable part of $Λ^{'} β$ .

$Λ_{0}$ is chosen, so that $C (Λ_{0}) = C (Λ) \cap C (X^{'})$ , which means that $Λ^{'} β = 0$ implies that $Λ_{0}^{'} β = 0$ but $Λ_{0}^{'} β$ is estimable, because $C (Λ_{0}) \subset C (X^{'})$ .

Theorem 3.3.6.

let $C (Λ_{0}) = C (Λ) \cap C (X^{'})$ and $C (U_{0}) = C (Λ_{0})^{⊥}$ . Then $C (X U) = C (X U_{0})$ . Thus $Λ^{'} β = 0$ and $Λ_{0}^{'} β = 0$ induce the same RM.

Proposition 3.3.7.

let $Λ_{0}^{'} β$ be estimable and $Λ \neq = 0$ . then $Λ^{'} β = 0 ⟹ C (X U) \neq = C (X)$ .

Corollary 3.3.8.

\mathcal{C}(\Lambda_0) = \mathcal{C}(\Lambda) ; \cap ; \mathcal{C}(X’) = {0 }

\Updownarrow

\mathcal{C}(XU) ; \cap ; \mathcal{C}(X)

A Generalized Test Procedure

Consider as below, whose column space is solvable.

$H_{0} : Λ^{'} β = d, d \in C (X^{'}), Λ^{'} b = d$

$ \begin{alignat}{2}

\Lambda ’ \beta =

\Lambda ’ b = d

; ; ; &\iff \Lambda ’ (\beta - b) &&= 0

&\iff (\beta - b) &&\perp \mathcal{C}(\Lambda)

&\iff (\beta - b) &&\in \mathcal{C}(U) ; ; ; ; ; ; &&\text{where } ; \mathcal{C}(U) = \mathcal{C}(\Lambda)^\perp

&\iff (\beta - b) &&= U_\gamma &&\exists \gamma

&\iff X\beta - Xb &&= XU_\gamma

& ; ; ; \Updownarrow

X\beta &= XU_\gamma + Xb, \

Y &= X \beta + \epsilon \ &= X U_\gamma + Xb + \epsilon \ &= X_0 \gamma + Xb + \epsilon, && && \text{where } ; X_0 = XU

\end{alignat}

if $Λ = X^{'} P$ , then $C (X_{0})_{C (X)}^{⊥} = C (MP)$ and its test statistics is

$ \begin{align}

F = \dfrac {\dfrac{(Y-Xb)‘M_{MP}(Y-Xb)}{r \Big(M_{MP} \Big)}} {\dfrac{(Y-Xb)‘(I-M)(Y-Xb)}{r \Big(I-M \Big)}}

\dfrac {\dfrac{(\Lambda ’ \hat \beta - d)’ \Big[ \Lambda’(X’X)^{-}\Lambda \Big]^- (\Lambda ’ \hat \beta - d)}{r(\Lambda)}} {MSE}

\sim F(?, ?, ?)

\end{align} $

Remark: (EXAMPLE 3.3.9.: pp.71–72, EXAMPLE 3.4.1.: pp.75)

If $Λ^{'} β = d$ , the same reduced model results if we take $Λ^{'} β = d_{0}$ , where $d_{0} = d + Λ^{'} ν$ and $ν ⊥ C (X^{'})$ . Note that, in this construction, if $Λ^{'} β = d$ is estimable, $d_{0} = d$ for any $ν$ .

Testing Single Degrees of Freedom in a Given Subspace

$ RM: Y=X_ 0 \gamma + \epsilon ; ; ; ; ; vs. ; ; ; ; ;

FM: Y=X \beta + \epsilon, ; ; ; ; ; with; ; \mathcal{C}(X_0) \subset \mathcal{C}(X)

let $M_{*} = M - M_{0}$ , consider $H_{0} : Λ^{'} β = 0$ .

if $Λ = X^{'} P$ , i.e. $Λ \in C (X^{'})$ , then $M_{*} = M_{MP}$ .

Proposition 3.3.2

Since $M M_{*} = M_{*}$ ,

\begin{align}

&\mathcal{C}(M - M_0) = \mathcal{C}(X_0){\mathcal{C}(X)}^\perp \equiv \mathcal{C}(XU){\mathcal{C}(X)}^\perp = \mathcal{C}(MP)

\Longrightarrow ; ; ;

&M \rho \in \mathcal{C}(M_\ast)

\Longrightarrow ; ; ;

&M \rho = M_\ast M \rho = M_\ast \rho

\Longrightarrow ; ; ;

&\rho ’ \hat \beta = \rho ’ M_\ast Y = \rho ’ M Y

\end{align} $

thus the test statistic for $H_{0} : Λ^{'} β = 0$ is

\dfrac{Y ’ M_\ast \rho ( \rho ’ M_\ast \rho )^{-1} \rho ’ M_\ast Y }{MSE}

\dfrac{\dfrac{(\rho ’ M_\ast Y)^2}{\rho ’ M_\ast \rho} }{MSE}

Breaking SS into Independent Components

Consider $X = (X_{0}, X_{1})$ . set

\begin{alignat}{2} &SSR(X_1 \vert X_0) &&\equiv Y ’ (M-M_0)Y && \tag{Sum of Squares for regression X1 after X0}\

&SSR(X) &&\equiv Y ’ MY \

&SSR(X_0) &&\equiv Y ’ M_0 Y \

&SSR(X) &&= SSR(X_0) &&+ SSR (X_1 \vert X_0)

\end{alignat} $

Note: if $ϵ \sim N (0, σ I)$ , then $SSR (X_{0}) ⊥ S s R (X_{1} ∣ X_{0})$ .

General Theory

Let $M$ and $M_{*}$ be the orthogonal projection operator into $C (X)$ and $C (X_{*})$ respectively. Then, with $C (X_{*}) \subset C (X)$ , $M_{*}$ defines a test statistic as below.

\dfrac {\dfrac{Y’ M_\ast Y}{r(M_\ast)}} {\dfrac{Y’ (I-M) Y}{r(I-M)}}

; ; ; \text{for RM}:Y = X_\ast \gamma + \epsilon

$ \begin{align}

&I-(M-M_\ast ) &&= (I-M) + M_\ast

&\mathcal{C}(M-M_\ast) &&:\tag{Estimation Space, under H0}

&\mathcal{C}(M_\ast) &&:\tag{Test Space, under H0}

&\mathcal{C} \Big(I - (M-M_\ast)\Big) &&:\tag{Error Space, under H0}

\end{align}

Using Gram-Schmidt procedure, let’s construct $M_{*}$ so that

M_\ast = RR’ = \sum_{i=1}^r R_iR_i ’ = \sum_{i=1}^r M_i, ; ; ; ; ; R=(R_1 , \cdots, R_r)

and $M_{i} M_{j} = 0$ for $i \neq = j$ . By Theorem 1.3.7,

$Y^{'} M_{i} Y ⊥ Y^{'} M_{j} Y ⟺ M_{i} M_{j} = 0$

Next, $Y^{'} M Y = \sum_{i = 1}^{r} Y^{'} M_{i} Y$ , therefore when $r (M_{i}) = 1$ ,

\dfrac {\dfrac{Y’M_i Y}{r(M_i)}} {\dfrac{Y’(I-M) Y}{r(I-M)}}

\sim F \Bigg( 1, r(I-M), \dfrac{1}{2 \sigma^2} \beta ’ X’ M_i X \beta \Bigg)

\begin{alignat}{2}

& && && &&\beta ’ X’ M_\ast X \beta ; ; &&= ; ; \sum_{i=1}^r \beta ’ X’ M_i X \beta && =0

; ; ;

&\iff && && \forall i ; ; : ; ; && \beta ’ X’ M_i X \beta && &&=0

&\iff && &&\forall i ; ; : ; ; &&R_i ’ X \beta && &&= 0

&\iff && && &&H_0 \text{ is true.}

\end{alignat} $

EXAMPLE 3.6.1.: Balanced design; pp.79–80
EXAMPLE 3.6.2.: Unbalanced design;pp.80–81

Two-Way ANOVA

$ \begin{alignat}{2} y_{ijk} &= \mu + \alpha_i + \eta_j &&+ \epsilon_{ijk} \tag{FM}

y_{ijk} &= \mu + \alpha_i &&+ \epsilon_{ijk} \tag{RM}

\end{alignat} $

$ \begin{align} M &= M_\mu + M_\alpha + M_\eta

Y’(M-M_0)Y &= R(\eta ; \Big \vert ; \alpha, ; \mu) \tag{1}

\end{align} $

Reduction in SSE, due to fitting $η_{j}$ ‘s after $μ$ and $α_{i}$ ‘s.

Next,

\begin{alignat}{2} y_{ijk} &= \mu + \alpha_i &&+ \epsilon_{ijk} \tag{FM}

y_{ijk} &= \mu &&+ \epsilon_{ijk} \tag{RM}

Y’(M_0-M_J)Y &= R(\alpha ; \Big \vert ; \mu)

Y’(M-M_J)Y &= R(\alpha, ; \eta ; \Big \vert ; \mu)

&= R(\eta ; \Big \vert ; \mu, ; \alpha) &&+ R(\alpha ; \Big \vert ; \mu)

\end{alignat}

In general,

\begin{alignat}{2}

R(\eta ; \Big \vert ; \alpha, ; \mu) &\not = R(\eta ; \Big \vert ; \mu)

R(\alpha ; \Big \vert ; \eta, ; \mu) & \not = R(\alpha ; \Big \vert ; \mu)

\end{alignat}

In paricular, for balanced design, if $C (X_{α}) ⊥ C (X_{η})$ ,

\begin{alignat}{2}

R(\eta ; \Big \vert ; \alpha, ; \mu) & = R(\eta ; \Big \vert ; \mu)

R(\alpha ; \Big \vert ; \eta, ; \mu) & = R(\alpha ; \Big \vert ; \mu)

\end{alignat}

Proposition 3.6.3.

\begin{alignat}{2}

R(\eta ; \Big \vert ; \alpha, ; \mu) & = R(\eta ; \Big \vert ; \mu)

; ; ; ; ; &&\iff ; ; ; ; ;

\mathcal{C}(X_1 - M_j) \perp \mathcal{C}(X_0 - M_j)

\text{that is}; ; ; ; ; ; ;

M_1 - M_J& = M-M_0

; ; ; ; ; &&\iff ; ; ; ; ;

(M_1 - M_J)(M_0 - M_J) = 0,

; ; ; ; ; \text{where} ; &&R(\eta ; \Big \vert ; \alpha, ; \mu) &&= Y’(M-M_0)Y

& && && R(\eta ; \Big \vert ; \mu) &&= Y’(M_1 -M_0)Y

\end{alignat}

Confidence Regions

$100 (1 - α) %$ Confidence Region(CR) for $Λ^{'} β$ consists of all the vectors $d$ satisfying the inequality

\dfrac {\dfrac{\Big[\Lambda ’ \hat \beta - d\Big]’ \Big[\Lambda ’ (X’X)^- \Lambda\Big]^- \Big[\Lambda ’ \hat \beta - d\Big]}{r(\Lambda)}} {MSE}

\le \Big( 1- \alpha, ; r(\Lambda), ; r(I-M) \Big)

These vectors form an ellipsoid in $r (Λ)$ -dimensional space.

For regression problems, if we take $P^{'} = (X^{'} X)^{- 1} X^{'}$ , then $Λ^{'} β = P^{'} Xβ = β = d$ .

The $100 (1 - α) %$ CR is

$ \begin{alignat}{2}

& \dfrac {\dfrac{\Big[\Lambda ’ \hat \beta - d\Big]’ \Big[\Lambda ’ (X’X)^- \Lambda\Big]^- \Big[\Lambda ’ \hat \beta - d\Big]}{r(\Lambda)}} {MSE}

; ; ; &&

; ; ; & \dfrac {\dfrac{\Big(\hat \beta - \beta \Big)’ \Big( X’X \Big)\Big(\hat \beta - \beta \Big)}

{p}} {MSE}

; ; ;

&&\le

; ; ; \Big( 1- \alpha, ; p, ; n-p \Big)

\end{alignat}

Tests for Generalized Least Squares Models

$ \begin{alignat}{4}

&Y &&= &&X \beta &&+ &&\epsilon ; ; ; ; ; &&vs. ; ; ; ; ; &&Y = &&X_0 \beta_0 &&+ &&\epsilon

, ; ; ; ; ; && \epsilon \sim N(0, ; \sigma^2 V)

\tag{1}

& && && && && && \Updownarrow

Q^{-1}&Y &&= Q^{-1} &&X \beta &&+ Q^{-1} &&\epsilon ; ; ; ; ; ; ; ; ; &&vs. ; ; ; ; ; Q^{-1} &&Y = Q^{-1} &&X_0 \beta_0 &&+ Q^{-1} &&\epsilon

, ; ; ; ; ; Q^{-1} && \epsilon \sim N(0, ; \sigma^2 I)

\tag{2}

\end{alignat} $

test (1) and (2) is equal.

Note: $C (Q^{- 1} X_{0}) \subset C (Q^{- 1} X)$ .

From Section 2.7,

$ \begin{align}

A &= X(X’V^{-1}X)^- X’ \ast V^{-1}

\ \

MSE &= \dfrac{Y’ (I-A)’ V^{-1} (I-A)Y}{n-r(X)}

\ \

A_0 &= X_0(X_0’V^{-1}X_0)^- X_0’ \ast V^{-1}

\end{align} $

Theorem 3.8.1

$ \begin{align}

\dfrac{\dfrac{Y’ (A-A_0) V^{-1} (A-A_0)Y}{r(X) - r(X_0 )}}{MSE} &\sim F \Big( r(X)-r(X_0), ; n-r(X) , ; \delta^2 \Big)

\ \

\delta^2 &= \dfrac{\beta ’ X’ (A-A_0) V^{-1} (A-A_0)X \beta}{2\sigma^2} \tag{1}

{\beta ’ X’ (A-A0) V^{-1} (A-A_0)X \beta} ; ; ; ; ; &\iff ; ; ; ; ; E(Y) \in \mathcal{C}(X_0) \tag{2}

\end{align} $

Theorem 3.8.2

let $Λ^{'} β$ be estimable. then the test statistic for $H_{0} : Λ^{'} β = 0$ is

$ \begin{align}

\dfrac{\dfrac{\hat \beta ’ \Lambda \Big[ \Lambda ’ (X’V^{-1}X)^- \Lambda \Big]^- \Lambda ’ \hat \beta}{r(\Lambda)}}{MSE} &\sim F \Big( r(\lambda), ; n-r(X) , ; \delta^2 \Big)

\ \

\delta^2 &= \dfrac{\beta ’ \Lambda \Big[ \Lambda ’ (X’V^{-1}X)^- \Lambda \Big]^- \Lambda ’ \beta}{2\sigma^2} \tag{1}

{\beta ’ \Lambda \Big[ \Lambda ’ (X’V^{-1}X)^- \Lambda \Big]^- \Lambda ’ \beta} ; ; ; ; ; &\iff ; ; ; ; ; \Lambda ’ \beta = 0\tag{2}

\end{align} $

Theorem 3.8.3

$ \begin{align}

\dfrac{Y’ (A-A_0) V^{-1} (A-A_0)Y}{\sigma^2} &\sim \chi^2\Big(r(x) - r(X_0), ; \delta^2 \Big)

\ \

\delta^2 &= \dfrac{\beta ’ X’ (A-A_0) V^{-1} (A-A_0)X \beta}{2\sigma^2},

\ \

\sigma^2 = 0 ; ; ; ; ; &\iff E(Y) \in \mathcal{C}(X_0)

\tag{1}

\dfrac{\hat \beta ’ \Lambda \Big[ \Lambda ’ (X’V^{-1}X)^- \Lambda \Big]^- \Lambda ’ \hat \beta}{2\sigma^2}

&\sim \chi^2 \Big( r(\Lambda) , ; \delta^2 \Big)

\ \

\delta^2 &= {\hat \beta ’ \Lambda \Big[ \Lambda ’ (X’V^{-1}X)^- \Lambda \Big]^- \Lambda ’ \hat \beta},

\ \

\sigma^2 = 0 ; ; ; ; ; &\iff \Lambda ’ \beta = 0 \tag{2}

\end{align} $

Quartz 4

Explorer

050. Testing

Testing

More About Models: Two approaches for linear model

Testing Models

A Generalized Test Procedure

Testing Linear Parametric Functions

Theoretical Complements

A Generalized Test Procedure

Testing Single Degrees of Freedom in a Given Subspace

Breaking SS into Independent Components

General Theory

Two-Way ANOVA

Confidence Regions

; ; ; &&

Tests for Generalized Least Squares Models

Graph View

Table of Contents