---

Multiple Regression

Forrest Young's Notes

Copyright © 1997-9 by Forrest W. Young.

---

Linear Combinations

A central aspect of Multiple regression is the idea of a linear combination. In this section we illustrate the idea of linear combinations with the following "dataset" of four observations on two variables:

 

3	2
3	5
1	3
5	1

• Algebraically, a list of numbers can represent:
• A numeric variable in a dataset (a column) -- This is often called a column vector. Thus, [3 3, 1, 5] is a column vector in these data.
• An observation in a dataset (a row) -- often called a row vector. Thus, [3, 5] is a row vector in these data.
• Geometrically, we can portray Linear combinations as shown in the following diagram:



Diagram: In this diagram, a list of P numbers can represent:

• A point in a P-dimensional space. The elements of the list are the coordinates of the point. The observation A is represented by a point.
• A directed line segment (a vector, an arrow or a ray) in a P-dimensional space. Observations B, C and D are represented by arrows.
• A linear combination of P variables. The elements of the list are the coefficients of the linear combination. This is not represented in this diagram. We'll get to it in a minute.


Note: There are two different meanings of the word "vector" here.

• Algebraically, a list of numbers.
• Geometrically, a directed line segment.