Some alternatives to least squares estimation in linear modelling: a thesis presented in partial fulfilment of the requirements for the degree of Master of Science in Statistics at Massey University

Abstract

The effects of non-standard conditions on the application of the Gauss-Markov Theorem are discussed and methods proposed in the literature for dealing with these effects are reviewed. The multicollinearity problem, which is typified by imprecise least squares estimation of parameters in a multiple linear regression and which arises when the vectors of the input or predictor variables are nearly linearly dependent, is focussed upon and a class of alternative biased estimators examined. In particular several members of the class of biased linear estimators or linear transformations of the Gauss-Markov least squares estimator are reviewed. A particular generalized ridge estimator is introduced and its relation to other techniques already existing in the literature is noted. The use of this estimator and the simple ridge regression estimator is illustrated on a small data set. Further comparisons of the estimator, the ridge estimator and other generalized ridge estimators are suggested.