# Bootstrap Regression

Two bootstrap techniques to get confidence intervals for regression coefficients are the case/pair bootstrap method and the residual bootstrap method.

## Case Bootstrap

The case or sometimes called pair is the response variables paired with the predictor variables $(y_i,x_i)$.

1. For B bootstrap samples
2. Take a sample with replacement of size $n$ from $(y_i,x_i)$ the corresponding bootstrap sample will be $(y_i^*, x_i^*)$.
3. Fit a model to $y_i^* = x_i^T \hat{\beta}$
4. Use the bootstrap regression coefficients to form confidence intervals for each coefficients.

## Residual Bootstrap

First fit a model of the form $y_i = \beta^0 + x^T_i\beta + \epsilon_i$

where $\epsilon \sim N(0,\sigma^2)$

1. For B bootstrap samples
2. Take a sample with replacement of size $n$ from $(e_i)$ the corresponding bootstrap sample will be $(e_i^*)$.
3. Fit  model to $y_i+e_i^* = x^T_i\hat{\beta}$
4. Use the bootstrap regression coefficients to form confidence intervals for each coefficients.

## R Example Residual Bootstrap

I didn’t set a seed so other runs might give different values due to random sampling.

t1* and t2* are the bootstrap estimates for the intercept and the dose term respectively.

These methods are good when there are few samples. The case bootstrap method is robust to outliers because some bootstrap samples will contain the outlier and will not which will capture the uncertainty due of the outliers.