050. Inference about Mean Vector

Inference about Mean Vector (wk3)

Overview

Recall: univariate case $x_1,\cdots ,x_n\stackrel{iid}{\sim }N(\mu ,{\sigma }^2)$

$H_0:\mu ={\mu }_0$

\begin{alignat*}{2} \text{test stat } &t &&=\tfrac{\bar X - \mu_0}{\tfrac{S}{\sqrt{n}}} &\overset{H_0}{\sim} t_{n-1} \\ &t^2 &&=\tfrac{(\bar X - \mu_0)^2}{\tfrac{S^2}{n}} &\overset{H_0}{\sim} F_{1, \; n-1} \end{alignat*}

reject $H_0$ if as below, which means upper $(100-\alpha )$th percentile.

---

$$\begin{array}{rl}\frac{(\bar{X}-{\mu }_0{)}^2}{\frac{S^2}{n}}=n(\bar{X}-{\mu }_0)\frac{1}{S^2}(\bar{X}-{\mu }_0)& >F_{1,n-1}(\alpha )\end{array}$$

therefore, with assumption $X_1,\cdots ,X_n\stackrel{iid}{\sim }{N}_p(\mu ,\Sigma )$,

$$H_0:\mu ={\mu }_0$$ $$\begin{array}{rl}\text{Hotelling’s }{T}^2{T}^2& =n(\bar{X}-{\mu }_0{)}^{\prime }{S}^{-1}(\bar{X}-{\mu }_0)\\ & \stackrel{H_0}{\sim }\frac{(n-1)p}{(n-p)}{F}_{p,n-p}\\ ⟺\frac{(n-p)}{(n-1)p}{T}^2& \stackrel{H_0}{\sim }{F}_{p,n-p}\end{array}$$

reject $H_0$, if $T^2>\frac{(n-1)p}{(n-p)}{F}_{p,n-p}(\alpha )$.

assumption check: $\pmb X_1 , \cdots, \pmb X_n \overset{iid}{\sim} N_p (\pmb \mu , \Sigma)$.

Remark

Hotelling’s T2T^2T2는

stat $T^2$는 측정 단위에 invariant.

proof)

let $Y_{p\times 1}=C_{p\times p}{X}_{p\times 1}+d_{p\times 1}$. then

$$\begin{array}{rl}\bar{Y}& =C\bar{X}+d\\ & \\ S_y& =CS{C}^{\prime }\\ & \\ {\mu }_y& =E(Y)\\ & =C\ast E(X)+d\\ & =C{\mu }_0+d\end{array}$$

therefore,

$$\begin{array}{rl}{T}^2& =n(\bar{Y}-{\mu }_y{)}^{\prime }{S}_y^{-1}(\bar{Y}-{\mu }_y)\\ & =n{\left[C(\bar{X}-{\mu }_0)\right]}^{\prime }(CS{C}^{\prime }{)}^{-1}\left[C(\bar{X}-{\mu }_0)\right]\\ & =n(\bar{X}-{\mu }_0{)}^{\prime }{C}^{\prime }(C^{\prime }{)}^{-1}{S}^{-1}(C{)}^{-1}C(\bar{X}-{\mu }_0)\\ & =n(\bar{X}-{\mu }_0{)}^{\prime }{S}^{-1}(\bar{X}-{\mu }_0)\end{array}$$

여기서 $C^{-1}$이 존재한다는게 뭔수로 보장되는거지?

---

여기서 $C^{-1}$의 존재를 보장하는 근거는 “측정 단위 변환”이 선형 변환이면서 가역적이어야 한다는 점 때문입니다.

---

1. 단위 변환이 선형변환 CCC로 표현되는 이유

• 예를 들어 cm → m, kg → g 같은 변환은 단순히 스케일링(scale matrix, 대각행렬) 입니다.

• 서로 다른 단위로 바꾼다고 해서 정보가 손실되면 안 되므로, 반드시 선형 + 전단사 변환이어야 합니다.

• 따라서 $C$ 는 항상 정칙 행렬(nonsingular)로 가정합니다.

---

2. $C$가 가역이어야 하는 이유

Hotelling’s $T^2$ 는

$$T^2=n(\bar{X}-{\mu }_0{)}^{\prime }{S}^{-1}(\bar{X}-{\mu }_0)$$

꼴인데, 단위변환 후에도 동일한 값이 나오려면

$$(CS{C}^{\prime }{)}^{-1}=(C^{\prime }{)}^{-1}{S}^{-1}{C}^{-1}$$

이 성립해야 합니다.
이 항등식은 $C$가 가역행렬일 때만 성립합니다.

즉, $C^{-1}$의 존재는 수학적으로 $T^2$ 불변성을 보이기 위한 필수 조건입니다.

---

3. 직관적인 해석

• 단위변환 행렬 $C$는 보통 양의 대각 원소만 가진 대각행렬 (예: cm→m이면 0.01, kg→g이면 1000)입니다.

• 대각행렬의 경우, 모든 원소가 0이 아닌 한 역행렬이 존재합니다.

• 그러므로 단위변환을 정상적으로 정의한다면 $C^{-1}$은 항상 보장됩니다.

---

✅ 결론:
C−1C^{-1}C−1의 존재는 측정 단위를 바꾸는 선형변환이 반드시 가역적이어야 한다는 가정에서 보장됩니다. 즉, 단위변환이 단순 스케일 조정(대각행렬)이라면 원소가 0일 수 없으므로 자동으로 invertible 합니다.

---

혹시 원하시면 제가 “만약 CCC가 비가역이면 T2T^2T2 불변성이 깨지는 구체적 반례”도 만들어드

---

1. Confidence Region

Confidence Region

region $R(X)$, is $100(1-\alpha )\%$ CR of

$$\begin{array}{rlrl}& P\left\{R(X)\text{ will cover the true }\theta \right\}& & =1-\alpha \\ & P\left\{n(\hat{X}-\mu {)}^{\prime }{S}^{-1}(\hat{X}-\mu )\le \frac{(n-1)p}{(n-p)}{F}_{p,n-p}(\alpha )\right\}& & =\end{array}$$

the inequality $n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )\le \frac{(n-1)p}{(n-p)}{F}_{p,n-p}(\alpha )$ will define a region $R(X)$.

The region is an ellipsoid centered at $\bar{X}$.

Testing $H_0:\mu ={\mu }_0$ at $\alpha =.05$ is equivalent to see whether ${\mu }_0$ falls within the CR.

• with ev ${\lambda }_1,\cdots ,{\lambda }_p$, evec $e_1,\cdots ,e_p$ of $S$,
  • CR Axis: $\pm \sqrt{\lambda }\sqrt{\frac{(n-1)p}{(n-p)}{F}_{p,n-p}(\alpha )}\ast e_i^{\prime }$
  • CR half-length: $\sqrt{\lambda }\sqrt{\frac{(n-1)p}{(n-p)}{F}_{p,n-p}(\alpha )}$

2. Simultaneous CI

let $X\sim N_p(\mu ,\Sigma )$, then linear combination $a^{\prime }X\sim N_p(a^{\prime }\mu ,a^{\prime }\Sigma a)$

$$\begin{array}{rl}t=\frac{\bar{X}-\mu }{S/\sqrt{n}}& \sim t_{n-1}\\ t=\frac{a^{\prime }\bar{X}-a^{\prime }\mu }{\sqrt{a^{\prime }Sa/n}}=\frac{\sqrt{n}(a^{\prime }\bar{X}-a^{\prime }\mu )}{\sqrt{a^{\prime }Sa}}& \sim t_{n-1}\end{array}\text{\hspace{1em}(recall: univariate)(MV)}$$

therefore, $100(1-\alpha )\%$ CI for $a^{\prime }\mu$ (at here, $a$is fixed) is $a^{\prime }\bar{X}\pm t_{n-1}\frac{\alpha }{2}\frac{\sqrt{a^{\prime }Sa}}{\sqrt{n}}$. This is not a simultaneous CI. let each $(a_1,a_2),(b_1,b_2)$ be CI for ${\mu }_1,{\mu }_2$. then simultaneous CI $(a_1,a_2),(b_1,b_2)$ has confidence $95\%\ast 95\%=90.25\%$. need a wider interval.

let rs $X_1,\cdots ,X_n\stackrel{iid}{\sim }{N}_p(\mu ,\Sigma )$.

then, simultaneously for all $a$, the interval $a^{\prime }\bar{X}\pm \sqrt{\frac{n-1}{n}\frac{p}{n-p}{F}_{p,n-p}(\alpha )a^{\prime }Sa}$ will contain $a^{\prime }\mu$ with probability $1-\alpha$.

$$\begin{array}{rl}\because 1-\alpha & =P\left[n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )\le (n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\right]\\ & =P\left[(a^{\prime }\bar{X}-a^{\prime }\mu {)}^{\prime }(a^{\prime }Sa{)}^{-1}(a^{\prime }\bar{X}-a^{\prime }\mu )\le \frac{1}{n}(n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\right]\\ & =P\left[(a^{\prime }\bar{X}-a^{\prime }\mu {)}^{\prime }(a^{\prime }\bar{X}-a^{\prime }\mu )\le \frac{1}{n}(n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\ast (a^{\prime }Sa)\right]\\ & =P\left[(a^{\prime }\bar{X}-a^{\prime }\mu {)}^2\le \frac{1}{n}(n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\ast (a^{\prime }Sa)\right]\\ & =P\left[-\sqrt{\frac{1}{n}(n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\ast (a^{\prime }Sa)}\le (a^{\prime }\bar{X}-a^{\prime }\mu )\le \sqrt{\frac{1}{n}(n-1)\frac{p}{n-p}{F}_{p,n-p}(\alpha )\ast (a^{\prime }Sa)}\right]\end{array}\text{\hspace{1em}(∵ Scalar)}$$

Simultaneous CI for ${\mu }_i-{\mu }_k$

let $a^{\prime }=[0,\cdots ,0,a_i,0,\cdots ,0,a_k,0,\cdots ,0]$. then as below, where $S=\left[\begin{array}{ccc}{S}_{11}& \cdots & S_{1p}\\ & \ddots & \\ S_{p1}& \cdots & S_{pp}\end{array}\right]$.

$$\begin{array}{rl}{a}^{\prime }\mu & ={\mu }_i-{\mu }_k\\ a^{\prime }Sa=S_{ii}-2S_{ik}+S_{k}k& \end{array}$$

therefore, the simultaneous CI for ${\mu }_i-{\mu }_k$, is $({\bar{x}}_i-{\bar{x}}_k)\pm \sqrt{\frac{n-1}{n}\frac{p}{n-p}{F}_{p,n-p}(\alpha )S_{ii}-2S_{ik}+S_{k}k}$.

---

at here, if we let $a^{\prime }=[1,0,\cdots ,0]$.

then

$$\begin{array}{rl}{a}^{\prime }\mu & ={\mu }_1\\ a^{\prime }Sa=S_{11}& \end{array}$$

therefore, the simultaneous CI for ${\mu }_1$, is ${\bar{x}}_1\pm \sqrt{\frac{n-1}{n}\frac{p}{n-p}{F}_{p,n-p}(\alpha )S_{11}}$.

---

3. Note: Bonferroni Multiple Comparison

Bonferroni’s CI, ${\bar{x}}_1\pm \left\{t_{n-1}\left(\frac{\alpha }{2p}\right)\right\}\sqrt{\frac{S_{1}1}{n}}$, is more precise (narrower) than simultaneous CI.

---

4. Large Sample Inferences about a Mean Vector

Recall mv CLT:

let $X_1,\cdots ,X_n\sim ?(\mu ,\Sigma )$ and for $n-p$ large. then

$$\begin{array}{rl}\sqrt{n}(\bar{X}-\mu )& \stackrel{d}{⟹}{N}_p(0,\Sigma )\\ n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )& \stackrel{d}{⟹}{\chi }_p^2\end{array}$$

---

when the sample size is large, the MVN assumption is less critical. therefore,

let $X_1,\cdots ,X_n\sim ?(\mu ,\Sigma )$.

$$H_0:\mu ={\mu }_0$$

when $n-p$ is large, the $H_0$ is rejected if $n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )>{\chi }_p^2(\alpha )$.

Note: $(n-1)\frac{p}{n-p}{F}_{p,n-p})\alpha \simeq {\chi }_p^2(\alpha )$, for large $n-p$.

---

• CI:

$$P\left[n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )\le {\chi }_p^2(\alpha )\right]=1-\alpha$$

the inequality $n(\bar{X}-\mu {)}^{\prime }{S}^{-1}(\bar{X}-\mu )\le {\chi }_p^2(\alpha )$ will define a region, which means, $100(1-\alpha )\%$ region.

• Simultaneous CI:

let $X_1,\cdots ,X_n\sim ?(\mu ,\Sigma )$ and for $n-p$ large. then

$\forall a$, $100(1-\alpha )\%$ simultaneous CI for $a^{\prime }\mu$ $=a^{\prime }\bar{X}\pm \sqrt{{\chi }_p^2(\alpha )}\sqrt{\frac{a^{\prime }Sa}{n}}$.

• Simultaneous CI for ${\mu }_i$

$${\bar{x}}_i\pm \sqrt{{\chi }_p^2(\alpha )}\sqrt{\frac{s_{ii}}{n}}$$

• Bonferroni’s CI for ${\mu }_i$

$${\bar{x}}_i\pm z_{\frac{\alpha }{2p}}\sqrt{\frac{s_{ii}}{n}}$$

- Bonferroni's CI is more precise. as also. 

1. Profile Analysis (wk4, 5)

if $X\sim N_p(\mu ,\Sigma )$, and the variables in $X$ are measured in the same unit, we may with to compare the means ${\mu }_1,\cdots ,{\mu }_p$ in $\mu$.

ex) repeated measure: a measurement is taken at the same experimental unit $p$ successive times.

A profile is a plot, connecting $(i,{\mu }_i),i=1,\cdots ,p$

---

Question: is the profile flat?

$$\begin{array}{rl}& H_0:{\mu }_1=\cdots ={\mu }_p\\ ⟺& H_0:C_1\mu =0,{\left[C_1\right]}_{(p-1)\times p}\\ ⟺& H_0:C_2\mu =0,{\left[C_2\right]}_{(p-1)\times p}\end{array}$$

---

if $X\sim N_p(\mu ,\Sigma )$, then $CX\sim N_{p-1}(C\mu ,C\Sigma C^{\prime })$, thus when $H_0:C\mu =0$ is true, then $C\bar{X}\sim N_{p-1}(C\mu ,C\Sigma C^{\prime })$.

test stat $T^2=n(C\bar{X}{)}^{\prime }(CS{C}^{\prime }{)}^{-1}(C\bar{X})\stackrel{H_0}{\sim }(n-1)\frac{p-1}{n-p+1}{F}_{p-1,n-p+1}$

reject $H_0$, if $T^2>(n-1)\frac{p-1}{n-p+1}{F}_{p-1,n-p+1}(\alpha )$.

**Note: $C_{(p-1)\times p}$ is not square, so there’s no inverse. thus $C$ in test stat doesn’t be canceled.

$H_0:C\mu =0$

where $C_{q\times p}(q\le p)$, and $rank(C)=q$. then

test stat $T^2=n(C\bar{X}{)}^{\prime }(CS{C}^{\prime }{)}^{-1}(C\bar{X})\stackrel{H_0}{\sim }(n-1)\frac{q}{n-q}{F}_{q,n-q}$

which means $p-1$ become $q$.

---

2. Test for Linear Trend

suppose $p$ variables are measured across equally spaced time periods. Also suppose $H_0:{\mu }_1=\cdots ={\mu }_p$ is rejected.

Question: Do the means fall onto a straight line?

$$\begin{array}{rl}& H_0:{\mu }_2-{\mu }_1=\cdots ={\mu }_p-{\mu }_{p-1}\\ ⟺& H_0:{\mu }_3-2{\mu }_2+{\mu }_1=0,\cdots ,{\mu }_p-2{\mu }_{p-1}+{\mu }_{p-2}=0\\ ⟺C_{(p-2)\times p},& H_0:C\mu =0\end{array}$$

at here, we acquire test stat $T^2\stackrel{H_0}{\sim }(n-1)\frac{p-2}{n-p+2}{F}_{p-2,n-p+2}$.

---

3. Inferences about a Covariance Matrix

let rs $X_1,\cdots ,X_n\stackrel{iid}{\sim }{N}_p(\mu ,\Sigma )$.

$$H_0:\Sigma ={\Sigma }_0$$

let $W=(n-1)S={\sum }_{i=1}^n(X_i-\bar{X})(X_i-\bar{X}{)}^{\prime }$. then

$${\Lambda }^{\ast }={\left(\frac{e}{v}\right)}^{\frac{pv}{2}}|{\Sigma }_0^{-1}W{|}^{\frac{v}{2}}\exp \left[-\frac{1}{2}tr({\Sigma }_0^{-1}W)\right],v=n-1$$

then calculate $L=-2ln{\Lambda }^{\ast }\stackrel{H_0}{\sim }$ function of ${\chi }^2$-distribution.

---

• Test for Sphericity (Test for no Correlation)

$$H_0:\Sigma ={\sigma }^{2}I$$

$\Lambda =\frac{|W|}{{\left[\frac{1}{p}tr(W)\right]}^p}\stackrel{H_0}{\sim }$ function of ${\chi }^2$-distribution.

---

• Test for Compound Symmetry

if $\Sigma =\left[\begin{array}{cccc}{\sigma }^2& \rho & \cdots & \rho \\ \rho & {\sigma }^2& & ⋮\\ ⋮& \rho & \ddots & \rho \\ \rho & \cdots & \rho & {\sigma }^2\end{array}\right]$, then $\Sigma$ has compound symmetry.

$H_0:\Sigma \text{ has compound symmetry.}$

Compute $\Lambda =\frac{|S|}{(S^2{)}^p(1-r{)}^{p-1}(1+(p-1)r)}$, where

• $S^2=\frac{1}{p}{\sum }_{i=1}^p{S}_{ii}$.
• $r=\frac{1}{p}\frac{2}{(1-p)S^2}{\sum }_{i<j}^p{S}_{ij}$.

reject $H_0$ if $Q>{\chi }_f^2(\alpha ),f=\frac{p(p+1)-4}{2}$

• $Q=-\frac{(N-1)-p(p+1{)}^2(2p-3)}{6(p-1)(p^2+p-4)}\ast \ln \Lambda$.