Squared deviations from the mean (SDM) result from squaring deviations. In probability theory and statistics, the definition of variance is either the expected value of the SDM (when considering a theoretical distribution) or its average value (for actual experimental data). Computations for analysis of variance involve the partitioning of a sum of SDM.

Background

An understanding of the computations involved is greatly enhanced by a study of the statistical value

${\displaystyle \operatorname {E} (X^{2})}$, where ${\displaystyle \operatorname {E} }$ is the expected value operator.

For a random variable ${\displaystyle X}$ with mean ${\displaystyle \mu }$ and variance ${\displaystyle \sigma ^{2}}$,

${\displaystyle \sigma ^{2}=\operatorname {E} (X^{2})-\mu ^{2}.}$^[1]

(Its derivation is shown here.) Therefore,

${\displaystyle \operatorname {E} (X^{2})=\sigma ^{2}+\mu ^{2}.}$

From the above, the following can be derived:

${\displaystyle \operatorname {E} \left(\sum \left(X^{2}\right)\right)=n\sigma ^{2}+n\mu ^{2},}$

${\displaystyle \operatorname {E} \left(\left(\sum X\right)^{2}\right)=n\sigma ^{2}+n^{2}\mu ^{2}.}$

Sample variance

The sum of squared deviations needed to calculate sample variance (before deciding whether to divide by n or n − 1) is most easily calculated as $${\displaystyle S=\sum x^{2}-{\frac {\left(\sum x\right)^{2}}{n}}.}$$

From the two derived expectations above the expected value of this sum is $${\displaystyle \operatorname {E} (S)=n\sigma ^{2}+n\mu ^{2}-{\frac {n\sigma ^{2}+n^{2}\mu ^{2}}{n}},}$$ which implies $${\displaystyle \operatorname {E} (S)=(n-1)\sigma ^{2}.}$$

This effectively proves the use of the divisor n − 1 in the calculation of an unbiased sample estimate of σ².

Partition — analysis of variance

In the situation where data is available for k different treatment groups having size n_i where i varies from 1 to k, then it is assumed that the expected mean of each group is

${\displaystyle \operatorname {E} (\mu _{i})=\mu +T_{i}}$

and the variance of each treatment group is unchanged from the population variance ${\displaystyle \sigma ^{2}}$.

Under the Null Hypothesis that the treatments have no effect, then each of the ${\displaystyle T_{i}}$ will be zero.

It is now possible to calculate three sums of squares:

Individual

${\displaystyle I=\sum x^{2}}$

${\displaystyle \operatorname {E} (I)=n\sigma ^{2}+n\mu ^{2}}$

Treatments

${\displaystyle T=\sum _{i=1}^{k}\left(\left(\sum x\right)^{2}/n_{i}\right)}$

${\displaystyle \operatorname {E} (T)=k\sigma ^{2}+\sum _{i=1}^{k}n_{i}(\mu +T_{i})^{2}}$

${\displaystyle \operatorname {E} (T)=k\sigma ^{2}+n\mu ^{2}+2\mu \sum _{i=1}^{k}(n_{i}T_{i})+\sum _{i=1}^{k}n_{i}(T_{i})^{2}}$

Under the null hypothesis that the treatments cause no differences and all the ${\displaystyle T_{i}}$ are zero, the expectation simplifies to

${\displaystyle \operatorname {E} (T)=k\sigma ^{2}+n\mu ^{2}.}$

Combination

${\displaystyle C=\left(\sum x\right)^{2}/n}$

${\displaystyle \operatorname {E} (C)=\sigma ^{2}+n\mu ^{2}}$

See also

• Absolute deviation
• Algorithms for calculating variance
• Errors and residuals
• Least squares
• Mean squared error
• Residual sum of squares
• Root mean square deviation
• Variance decomposition of forecast errors

References