模型:$Y_i=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}+\varepsilon_i$
假設:$\varepsilon_i\stackrel{iid}{\sim}N(0,\sigma^2)$
母體迴歸線:$E(Y_i)=\beta_0+\beta_1X_{1i}+\beta_2X_{2i}$
樣本迴歸線:$\hat{Y}=\hat{\beta_0}+\hat{\beta_1}X_{1}+\hat{\beta_2}X_{2}$
正規方程式
$\left\{\begin{array}{}n\hat{\beta_0}&+\sum{}X_{1i} \hat{\beta_1}&+\sum{}X_{2i} \hat{\beta_2}&=\sum{}Y_i \\ \sum{}X_{1i}\hat{\beta_0}&+\sum{}X_{1i}^2\hat{\beta_1}&+\sum{}X_{1i}X_{2i}\hat{\beta_2}&=\sum{}X_{1i}Y_i\\ \sum{}X_{2i}\hat{\beta_0}&+\sum{}X_{1i}X_{2i}\hat{\beta_1}&+\sum{}X_{2i}^2\hat{\beta_2}&=\sum{}X_{2i}Y_i
\end{array}
\right.$
正規方程式之解
$\left\{\begin{array}{}
\hat{\beta_1}&=\cfrac{SS_{1Y}SS_2-SS_{2Y}SS_{12}}{SS_1SS_2-SS^2_{12}}\\ \hat{\beta_2}&=\cfrac{SS_{2Y}SS_1-SS_{1Y}SS_{12}}{SS_1SS_2-SS^2_{12}} \\ \hat{\beta_0}&=\overline{Y}-\hat{\beta_1}\overline{X_1}-\hat{\beta_2}\overline{X_2}
\end{array}
\right.$
分子分母皆$SS_1SS_2-SS_{12}SS_{21}$ ，$\hat{\beta_1}$時分子$Y$替換$SS_1$為$SS_{1Y}$，替換$SS_{12}$為$SS_{Y2}=SS_{2Y}$。
同理，$\hat{\beta_2}$時分子$Y$替換$SS_2$為$SS_{2Y}$，替換$SS_{21}$為$SS_{Y1}=SS_{1Y}$。
$SS_{12}=SS_{X_1X_2}=\sum X_{1i}X_{2i}-\cfrac{(\sum X_{1i})(\sum X_{2i})}{n}$
$SS_{1Y}=SS_{X_1Y}=\sum X_{1i}Y_i-\cfrac{(\sum X_{1i})(\sum Y_i)}{n}$
$SS_{2Y}=SS_{X_2Y}=\sum X_{2i}Y_i-\cfrac{(\sum X_{2i})(\sum Y_i)}{n}$
$SS_1=\sum X_{1i}^2-\cfrac{(\sum X_{1i})^2}{n}$
$SS_2=\sum X_{2i}^2-\cfrac{(\sum X_{2i})^2}{n}$
$SS_Y=\sum Y_{i}^2-\cfrac{(\sum Y_{i})^2}{n}$
抽樣分配
$\hat{\beta_1}\sim N(\beta_1,\cfrac{SS_2}{SS_1SS_2-SS^2_{12}}\sigma^2)$
$\hat{\beta_2}\sim N(\beta_2,\cfrac{SS_1}{SS_1SS_2-SS^2_{12}}\sigma^2)$
偏相關係數
圖解
$SSR(X_1|X_2)=SSR(X_1,X_2)-SSR(X_2)=SSE(X_2)-SSE(X_1,X_2)$
$r_{Y1\cdot2}^2=\cfrac{SSR(X_1|X_2)}{SSE(X_2)}=\cfrac{SSR(X_1|X_2)=SSR(去掉X_2)}{SST(去掉X_2)}$
視為去掉$X_2$因素，$SSE(X_2)=SST(去掉X_2)$
$r_{Y2\cdot1}^2=\cfrac{SSR(X_2|X_1)}{SSE(X_1)}$
同理可得:$r_{Y12\cdot34}^2=\cfrac{SSR(X_1,X_2|X_3,X_4)}{SSE(X_3,X_4)}$
$SST$和自變數個數無關。
複判定係數
$SSR(X_1,X_2,\cdots,X_k)=\displaystyle\sum_{i=1}^{k}\hat{\beta_i}SS_{iY}$
$R_{Y12\cdots k}^2=\cfrac{SSR(X_1,X_2,\cdots,X_k)}{SST}$
調整後之複判定係數
$R_{adj}^2=1-\cfrac{MSE}{MST}=1-\cfrac{\frac{SSE}{n-k-1}}{\frac{SST}{n-1}}$
假設檢定
1.檢定其中之一迴歸係數是否顯著?
$H_0:\beta_j=0$, ，$\left\{\begin{array}{c}
H_1:\beta_j\neq0\\ H_1:\beta_j>0 \\ H_1:\beta_j<0 \end{array}\right.$, $j=1,2,\cdots,k$
$T.S. :$ $T=\cfrac{\hat{\beta_j}-0}{S(\beta_j)}\sim t_{n-k-1}$
2.$H_0:\beta_j=0$，$H_1:\beta_j\neq0$ ，$j=1,2,\cdots,k$
$T.S. :$ $F=\cfrac{\frac{SSR(X_j|X_1,\cdots,X_{j-1},X_{j+1},\cdots,X_k)}{1}}{\frac{SSE(X_1,\cdots,X_k)}{n-k-1}}\sim F_{\alpha(1,n-k-1)}$
2.檢定多個迴歸係數是否顯著?
只可檢定$\beta_1=\beta_2=\cdots=\beta_j=0$ ,$j\leq k$，和雙尾檢定。
$H_0:\beta_1=\beta_2=\cdots=\beta_j=0$,$j\leq k$ $H_1:\beta_i不全為0$
$T.S. :$ $F=\cfrac{\frac{SSR(X_{目標}|X_{剩下})}{目標數}}{\frac{SSE(X_{所有})}{n-k-1}}\sim F_{(目標數,n-k-1)}$
