Linear regression

related topics
{math, number, function}
{rate, high, increase}
{disease, patient, cell}
{theory, work, human}
{math, energy, light}
{company, market, business}

In statistics, linear regression is an approach to modeling the relationship between a scalar variable y and one or more variables denoted X. In linear regression, models of the unknown parameters are estimated from the data using linear functions. Such models are called linear models. Most commonly, linear regression refers to a model in which the conditional mean of y given the value of X is an affine function of X. Less commonly, linear regression could refer to a model in which the median, or some other quantile of the conditional distribution of y given X is expressed as a linear function of X. Like all forms of regression analysis, linear regression focuses on the conditional probability distribution of y given X, rather than on the joint probability distribution of y and X, which is the domain of multivariate analysis.

Linear regression was the first type of regression analysis to be studied rigorously, and to be used extensively in practical applications. This is because models which depend linearly on their unknown parameters are easier to fit than models which are non-linearly related to their parameters and because the statistical properties of the resulting estimators are easier to determine.

Linear regression has many practical uses. Most applications of linear regression fall into one of the following two broad categories:

  • If the goal is prediction, or forecasting, linear regression can be used to fit a predictive model to an observed data set of y and X values. After developing such a model, if an additional value of X is then given without its accompanying value of y, the fitted model can be used to make a prediction of the value of y.
  • Given a variable y and a number of variables X1, ..., Xp that may be related to y, then linear regression analysis can be applied to quantify the strength of the relationship between y and the Xj, to assess which Xj may have no relationship with y at all, and to identify which subsets of the Xj contain redundant information about y, thus once one of them is known, the others are no longer informative.

Linear regression models are often fitted using the least squares approach, but they may also be fitted in other ways, such as by minimizing the “lack of fit” in some other norm, or by minimizing a penalized version of the least squares loss function as in ridge regression. Conversely, the least squares approach can be used to fit models that are not linear models. Thus, while the terms “least squares” and linear model are closely linked, they are not synonymous.

Contents

Full article ▸

related documents
Maximum likelihood
Central limit theorem
Entropy (information theory)
Hash function
Huffman coding
Interval (mathematics)
Absolute value
Simplex
Proofs of Fermat's little theorem
Frame problem
Factorial
Compass and straightedge constructions
Sequence alignment
Probability theory
Monte Carlo method
Lua (programming language)
Gaussian elimination
Numerical analysis
Cardinal number
RSA
Prime number theorem
Group action
Kernel (algebra)
Dynamic programming
Multivariate normal distribution
Abelian group
Computable number
Power law
Fundamental group
Hyperreal number