Multiple Regression

Forrest Young's Notes

Copyright © 1997-9 by Forrest W. Young.

Overview of Multiple Regression

Multiple Regression
Multiple Regression is a statistical technique that is used to measure and describe the function relating two (or more) predictor (independent) variables to a single response (dependent) variable.
Multiple Correlation Coefficient
The Multiple Correlation Coefficient tells us about the strength of the relationship of a combination of the several predictor variables with the response variable. The square of the multiple correlation tells us the proportion of the response variable's variance that is shared with the predictor variables variance.
Multiple Regression
Multiple regression tells us about the nature of the function that relates a set of predictor variables to a single response variable. OLS (ordinary least squares) multiple regression assumes the function is alinear function (straight line, flat plane, etc), and fits the function so that the relationship is is a least squares relationship. The function is called the regression function.
The Regression Line:
Multiple Regression Analysis fits a linear function to the relationship between the two variables. The line that is fit to the relationship has certain properties:
  • The regression function is as close as possible, in a specific average least squares sense, to all of the points.
  • The regression function identifies the "central tendency" of the relationship between the two variables, just as the mean identifies the "central tendency" of a single variable.
  • The regression function provides a simplified description --- a model --- of the relationship between the two variables.
  • The regression function gives us a way to predict values for the response variable from values of the predictor variable.
Regression Equation for a Linear Relationship
A linear relationship of n predictor variables, denoted
X1, X2, ... Xn
to a single response variable, denote
Y
is described by the linear equation involving several variables. The general linear equation is:
Y = a + b1X1 + b2X2 + ... + bnXn
This equation shows that any linear relationship  can be described by its:
  • Intercept: The linear combination of the X's is zero. Usually we don't interpret the intercept.
  • Slopes: The slope for each predictor variable is denoted by the various b's in the equation. The slope specifies how much the variable Y will change when the particular X changes by one unit.