Why variance in model and residual add up the way they do?

a. Standard deviation of a centered score s_x=\sqrt{\frac{\sum{x_i^2}}{N-1}}.

Hence if one thinks of the variable as a N-dimensional vector then the standard deviation corresponds roughly to the length of the vector.

b. When you regress y onto x, there is a component of y \widehat{y} lying along x, and then there is the remaining component e which is orthogonal to x.

c. \widehat{y}, e and y form a right-angled triangle with y being the hypotenuse. Hence,

|y|^2 = |\widehat{y}|^2 + |e|^2.

In plain english, sum of squares of dependent variable can be partitioned into squares that sum to the total. End of story.

Long live Pythagoras!

p.s. and the author of the book “The Geometry of Multivariate Statistics”, Thomas D. Wickens.

This entry was posted in Books, Notes and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s