Correlation: Encyclopedia II - Correlation - Pearson's product-moment coefficient

Correlation - Pearson's product-moment coefficient

Correlation - Mathematical properties

The correlation ρ_{X, Y} between two random variables X and Y with expected values μ_X and μ_Y and standard deviations σ_X and σ_Y is defined as:

Since μ_X = E(X), σ_X² = E(X²) − E²(X) and likewise for Y, we may also write

The correlation is defined only if both standard deviations are finite and both of them are nonzero. It is a corollary of the Cauchy-Schwarz inequality that the correlation cannot exceed 1 in absolute value.

The correlation is 1 in the case of an increasing linear relationship, −1 in the case of a decreasing linear relationship, and some value in between in all other cases, indicating the degree of linear dependence between the variables. The closer the coefficient is to either −1 or 1, the stronger the correlation between the variables.

If the variables are independent then the correlation is 0, but the converse is not true because the correlation coefficient detects only linear dependencies between two variables. Here is an example: Suppose the random variable X is uniformly distributed on the interval from −1 to 1, and Y = X². Then Y is completely determined by X, so that X and Y are dependent, but their correlation is zero; they are uncorrelated. However, in the special case when X and Y are jointly normal, independence is equivalent to uncorrelatedness.

Correlation - The sample correlation

If we have a series of n  measurements of X  and Y  written as x_i  and y_i  where i = 1, 2, ..., n, then the Pearson product-moment correlation coefficient can be used to estimate the correlation of X  and Y . The Pearson coefficient is also known as the "sample correlation coefficient". It is especially important if X  and Y  are both normally distributed. The Pearson correlation coefficient is then the best estimate of the correlation of X  and Y . The Pearson correlation coefficient is written:

where and are the sample means of x_i  and y_i , s_x  and s_y  are the sample standard deviations of x_i  and y_i  and the sum is from i = 1 to n. As with the population correlation, we may rewrite this as

Again, as is true with the population correlation, the absolute value of the sample correlation must be less than or equal to 1.

The sample correlation coefficient is the fraction of the variance in y_i  that is accounted for by a linear fit of x_i  to y_i . This is written

where σ_y|x²  is the square of the error of a linear fit of y_i  to x_i  by the equation y = a + bx.

and σ_y²  is just the variance of y

Note that since the sample correlation coefficient is symmetric in x_i  and y_i , we will get the same value for a fit of x_i  to y_i :

This equation also gives an intuitive idea of the correlation coefficient for higher dimensions. Just as the above described sample correlation coefficient is the fraction of variance accounted for by the fit of a 1-dimensional linear submanifold to a set of 2-dimensional vectors (x_i , y_i ), so we can define a correlation coefficient for a fit of an m-dimensional linear submanifold to a set of n-dimensional vectors. For example, if we fit a plane z = a + bx + cy  to a set of data (x_i , y_i , z_i ) then the correlation coefficient of z  to x  and y  is

Adapted from the Wikipedia article "Pearson's product-moment coefficient", under the G.N U Free Docmentation License. Please also see http://en.wikipedia.org/wiki