(迴歸)複迴歸分析

模型:$Y_i=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}+\varepsilon_i$

假設:$\varepsilon_i\stackrel{iid}{\sim}N(0,\sigma^2)$

母體迴歸線:$E(Y_i)=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}$

樣本迴歸線:$\hat{Y}=\hat{\beta_0}+\hat{\beta_1}X_{1}+\hat{\beta_2}X_{2}$

正規方程式

$\left\{\begin{array}{}n\hat{\beta_0}&+\sum{}X_{1i} \hat{\beta_1}&+\sum{}X_{2i} \hat{\beta_2}&=\sum{}Y_i \\ \sum{}X_{1i}\hat{\beta_0}&+\sum{}X_{1i}^2\hat{\beta_1}&+\sum{}X_{1i}X_{2i}\hat{\beta_2}&=\sum{}X_{1i}Y_i\\ \sum{}X_{2i}\hat{\beta_0}&+\sum{}X_{1i}X_{2i}\hat{\beta_1}&+\sum{}X_{2i}^2\hat{\beta_2}&=\sum{}X_{2i}Y_i \end{array} \right.$

正規方程式之解

$\left\{\begin{array}{} \hat{\beta_1}&=\cfrac{SS_{1Y}SS_2-SS_{2Y}SS_{12}}{SS_1SS_2-SS^2_{12}}\\ \hat{\beta_2}&=\cfrac{SS_{2Y}SS_1-SS_{1Y}SS_{12}}{SS_1SS_2-SS^2_{12}} \\ \hat{\beta_0}&=\overline{Y}-\hat{\beta_1}\overline{X_1}-\hat{\beta_2}\overline{X_2} \end{array} \right.$

分子分母皆$SS_1SS_2-SS_{12}SS_{21}$ ，$\hat{\beta_1}$時分子$Y$替換$SS_1$為$SS_{1Y}$，替換$SS_{12}$為$SS_{Y2}=SS_{2Y}$。

同理，$\hat{\beta_2}$時分子$Y$替換$SS_2$為$SS_{2Y}$，替換$SS_{21}$為$SS_{Y1}=SS_{1Y}$。

$SS_{12}=SS_{X_1X_2}=\sum X_{1i}X_{2i}-\cfrac{(\sum X_{1i})(\sum X_{2i})}{n}$

$SS_{1Y}=SS_{X_1Y}=\sum X_{1i}Y_i-\cfrac{(\sum X_{1i})(\sum Y_i)}{n}$

$SS_{2Y}=SS_{X_2Y}=\sum X_{2i}Y_i-\cfrac{(\sum X_{2i})(\sum Y_i)}{n}$

$SS_1=\sum X_{1i}^2-\cfrac{(\sum X_{1i})^2}{n}$

$SS_2=\sum X_{2i}^2-\cfrac{(\sum X_{2i})^2}{n}$

$SS_Y=\sum Y_{i}^2-\cfrac{(\sum Y_{i})^2}{n}$

抽樣分配

$\hat{\beta_1}\sim N(\beta_1,\cfrac{SS_2}{SS_1SS_2-SS^2_{12}}\sigma^2)$

$\hat{\beta_2}\sim N(\beta_2,\cfrac{SS_1}{SS_1SS_2-SS^2_{12}}\sigma^2)$

偏相關係數

圖解

$SSR(X_1|X_2)=SSR(X_1,X_2)-SSR(X_2)=SSE(X_2)-SSE(X_1,X_2)$

$r_{Y1\cdot2}^2=\cfrac{SSR(X_1|X_2)}{SSE(X_2)}=\cfrac{SSR(X_1|X_2)=SSR(去掉X_2)}{SST(去掉X_2)}$

視為去掉$X_2$因素，$SSE(X_2)=SST(去掉X_2)$

$r_{Y2\cdot1}^2=\cfrac{SSR(X_2|X_1)}{SSE(X_1)}$

同理可得:$r_{Y12\cdot34}^2=\cfrac{SSR(X_1,X_2|X_3,X_4)}{SSE(X_3,X_4)}$

$SST$和自變數個數無關。

複判定係數

$SSR(X_1,X_2,\cdots,X_k)=\displaystyle\sum_{i=1}^{k}\hat{\beta_i}SS_{iY}$

$R_{Y12\cdots k}^2=\cfrac{SSR(X_1,X_2,\cdots,X_k)}{SST}$

調整後之複判定係數

$R_{adj}^2=1-\cfrac{MSE}{MST}=1-\cfrac{\frac{SSE}{n-k-1}}{\frac{SST}{n-1}}$

假設檢定

1.檢定其中之一迴歸係數是否顯著?

$H_0:\beta_j=0$, ，$\left\{\begin{array}{c} H_1:\beta_j\neq0\\ H_1:\beta_j>0 \\ H_1:\beta_j<0 \end{array}\right.$,  $j=1,2,\cdots,k$

$T.S. :$ $T=\cfrac{\hat{\beta_j}-0}{S(\beta_j)}\sim t_{n-k-1}$

2.$H_0:\beta_j=0$，$H_1:\beta_j\neq0$ ，$j=1,2,\cdots,k$

$T.S. :$ $F=\cfrac{\frac{SSR(X_j|X_1,\cdots,X_{j-1},X_{j+1},\cdots,X_k)}{1}}{\frac{SSE(X_1,\cdots,X_k)}{n-k-1}}\sim F_{\alpha(1,n-k-1)}$

2.檢定多個迴歸係數是否顯著?

只可檢定$\beta_1=\beta_2=\cdots=\beta_j=0$ ,$j\leq k$，和雙尾檢定。

$H_0:\beta_1=\beta_2=\cdots=\beta_j=0$,$j\leq k$ $H_1:\beta_i不全為0$

$T.S. :$ $F=\cfrac{\frac{SSR(X_{目標}|X_{剩下})}{目標數}}{\frac{SSE(X_{所有})}{n-k-1}}\sim F_{(目標數,n-k-1)}$