Statistics and Probability

1. Introduction

This chapter studies how two variables move together and how to reason about uncertain events. Correlation measures the strength and direction of a linear relationship between two variables, regression finds the line that best predicts one variable from the other, and the probability sections extend the basic laws to conditional probability and to the binomial distribution, the model for repeated independent trials with two outcomes.

---

2. Correlation

Two variables are correlated when they tend to change together. The correlation is positive if they increase together, negative if one increases as the other decreases, and there is no linear correlation when no such tendency exists.

Karl Pearson's coefficient of correlation

For $n$ paired observations $(x_i, y_i)$, Karl Pearson's coefficient is

$$r = \frac{\sum (x-\bar{x})(y-\bar{y})}{\sqrt{\sum (x-\bar{x})^2},\sqrt{\sum (y-\bar{y})^2}},$$

which in the convenient computing form is

$$r = \frac{n\sum xy - \sum x \sum y}{\sqrt{n\sum x^2 - (\sum x)^2},\sqrt{n\sum y^2 - (\sum y)^2}}.$$

Its value always lies in $-1 \le r \le 1$. Values near $+1$ or $-1$ indicate strong linear association, and $r=0$ indicates no linear relationship. The coefficient is independent of the choice of origin and scale, and it is symmetric in $x$ and $y$.

Worked example. For the pairs $(1,2),(2,4),(3,5),(4,4),(5,5)$ we compute $\sum x = 15$, $\sum y = 20$, $\sum xy = 66$, $\sum x^2 = 55$, $\sum y^2 = 86$, with $n=5$. Then

$$ \begin{aligned} r &= \frac{5(66) - (15)(20)}{\sqrt{5(55) - 15^2},\sqrt{5(86) - 20^2}} \ &= \frac{330 - 300}{\sqrt{50},\sqrt{30}} \ &= \frac{30}{\sqrt{1500}} \approx 0.77 \end{aligned} $$

a fairly strong positive correlation.