An Algorithmic Approach to Linear Regression

We analyze a procedure how to build a high quality linear regression model. We start with an overview of the desirable properties, for example robustness and limited pairwise multicollinearity, of the linear regression model. We discuss the problem that the current approaches are not capable to find a linear regression model with the desirable properties. Therefore, our goal is to find a procedure that produces a linear regression model which achieves the desirable properties in a reasonable amount of time. We present an algorithmic approach in which the desirable properties are modeled as constraints and through penalties in the objective function of a Mixed Integer Quadratic Optimization (MIQO) model. The performance of the algorithm is shown on both real and synthetic data sets, and is compared with the widely used Lasso approach from Tibshirani (1996). Lastly, we extent the MIQO model with a heuristically chosen subset of interaction terms and compare the performance of the