Bootstrap Regression

Two bootstrap techniques to get confidence intervals for regression coefficients are the case/pair bootstrap method and the residual bootstrap method.

Case Bootstrap

 

The case or sometimes called pair is the response variables paired with the predictor variables (y_i,x_i).

  1. For B bootstrap samples
  2. Take a sample with replacement of size n from (y_i,x_i) the corresponding bootstrap sample will be (y_i^*, x_i^*).
  3. Fit a model to y_i^* = x_i^T \hat{\beta}
  4. Use the bootstrap regression coefficients to form confidence intervals for each coefficients.

Residual Bootstrap

First fit a model of the form y_i = \beta^0 + x^T_i\beta + \epsilon_i

where \epsilon \sim N(0,\sigma^2)

  1. For B bootstrap samples
  2. Take a sample with replacement of size n from (e_i) the corresponding bootstrap sample will be (e_i^*).
  3. Fit  model to y_i+e_i^* = x^T_i\hat{\beta}
  4. Use the bootstrap regression coefficients to form confidence intervals for each coefficients.

R Example Case Bootstrap

R Example Residual Bootstrap

I didn’t set a seed so other runs might give different values due to random sampling.

t1* and t2* are the bootstrap estimates for the intercept and the dose term respectively.

These methods are good when there are few samples. The case bootstrap method is robust to outliers because some bootstrap samples will contain the outlier and will not which will capture the uncertainty due of the outliers.