Multiple Regression
Forrest Young's Notes
Copyright © 1997-9 by Forrest W. Young.
Review of Simple Regression
Simple Regression, the previous topic, is very closely related to Multiple Regression.
- Simple Regression
- Simple Regression (just called "Regression", up to now) is a statistical technique that is used to measure and describe the function relating a predictor (independent) variable to a response (dependent) variable.
- Multiple Regression
- Multiple Regression is a statistical technique that is used to measure and describe the function relating two (or more) predictor (independent) variables to one response (dependent) variable.
- Correlation Coefficient:
- Correlation tells us about the strength (and shape) of the relationship between two variables. The square of the correlation tells us the proportion of the variables' variance that is shared between them.
- Multiple Correlation Coefficient:
- The Multiple Correlation Coefficient tells us about the strength of the relationship of a combination of the several predictor variables with the response variable. The square of the multiple correlation tells us the proportion of the response variable's variance that is shared with the predictor variables variance.
- Simple Regression:
- Regression tells us about the nature of the function relating the two variables. For linear regression, which is what we consider here, regression fits a straight line, called the regression line, to the data so that the line best describes their relationship.
- Multiple Regression
- Multiple regression tells us about the nature of the function that relates a set of predictor variables to a single response variable. OLS (ordinary least squares) multiple regression assumes the function is alinear function (straight line, flat plane, etc), and fits the function so that the relationship is is a least squares relationship. The function is called the regression function.
- The Regression Line:
- Regression Analysis fits a linear function to the relationship between the two variables. The line that is fit to the relationship has certain properties:
- The regression function is as close as possible, in a specific average least squares sense, to all of the points.
- The regression function identifies the "central tendency" of the relationship between the two variables, just as the mean identifies the "central tendency" of a single variable.
- The regression function provides a simplified description --- a model --- of the relationship between the two variables.
- The regression function gives us a way to predict values for the response variable from values of the predictor variable.
- Regression Equation for a Linear Relationship
- A linear relationship of n predictor variables, denoted
- X1, X2, ... Xn
- to a single response variable, denoted
- Y
- is described by the linear equation involving several variables. The general linear equation is:
- Y = a + b1X1 + b2X2 + ... + bnXn
- This equation shows that any linear relationship can be described by its:
- Intercept: The linear combination of the X's is zero. Usually we don't interpret the intercept.
- Slopes: The slope for each predictor variable is denoted by the various b's in the equation. The slope specifies how much the variable Y will change when the particular X changes by one unit.