Let X1,X2,...,XnX_1, X_2, ..., X_n be a set of i.i.d.i.i.d. random variables with mean μ\mu and varaince σ2\sigma^2. Define X‾=1n∑iXi\overline X = \frac{1}{n} \sum_i X_i. Then

∑i(Xi−X‾)2=∑i(Xi−μ−(X‾−μ))2=∑i(Xi−μ)2−2∑i(Xi−μ)(X‾−μ)+n(X‾−μ)2=∑i(Xi−μ)2−n(X‾−μ)2 \begin{aligned} \sum_i (X_i - \overline X)^2 &= \sum_i (X_i - \mu - (\overline X - \mu))^2 \\ &= \sum_i (X_i - \mu)^2 - 2 \sum_i (X_i - \mu)(\overline X - \mu) + n (\overline X - \mu)^2 \\ &= \sum_i (X_i - \mu)^2 - n (\overline X - \mu)^2 \end{aligned}

Define S=∑iXiS = \sum_i X_i. Note that SS is a random variable with mean nμn\mu and variance nσ2n\sigma^2. Therefore,

E[∑i(Xi−X‾)2]=∑iE[(Xi−μ)2]−nE[(X‾−μ)2]=nσ2−1nE[(S−nμ)2]=(n−1)σ2 \begin{aligned} E[ \sum_i (X_i - \overline X)^2 ] &= \sum_i E[(X_i - \mu)^2] - n E[(\overline X - \mu)^2] \\ &= n \sigma^2 - \frac{1}{n} E \left[ (S - n\mu)^2 \right] \\ &= (n - 1) \sigma^2 \end{aligned}

Another way of looking this is that,

1n∑i(Xi−X‾)2=1n(∑iXi2−2X‾(∑iXi)+n(X‾)2)=1n(∑iXi2−n(X‾)2)=1n∑iXi2−X‾2=1n∑iXi2−μ2−(X‾2−μ2) \begin{aligned} \frac{1}{n}\sum_i (X_i - \overline X)^2 &= \frac{1}{n} (\sum_{i} X_i^2 - 2 \overline X (\sum_{i} X_i) + n(\overline X)^2) \\ &= \frac{1}{n}( \sum_{i} X_i^2 - n(\overline X)^2 ) \\ &= \frac{1}{n} \sum_{i} X_i^2 - \overline X^2 \\ &= \frac{1}{n} \sum_{i} X_i^2 - \mu^2 - (\overline X^2 - \mu^2) \end{aligned}

Therefore,

E[1n∑i(Xi−X‾)2]=σ2−1nσ2 \begin{aligned} E[\frac{1}{n}\sum_i (X_i - \overline X)^2] &= \sigma^2 - \frac{1}{n} \sigma^2 \end{aligned}

We can also prove this via linear algebra. Let X=(X1,X2,...,Xn)X = (X_1, X_2, ..., X_n) be a nn-dimension vector. Also, denote e=(1,1,...,1)e = (1, 1, ..., 1). Then

X‾=1neTX∑i(Xi−X‾)2=(X−1neeTX)T(X−1neeTX)=XT(I−1neeT)T(I−1neeT)X=XT(I−1neeT)(I−1neeT)X=XT(I−21neeT+1n2eeTeeT)X=XT(I−21neeT+1neeT)X=XT(I−1neeT)X=Tr[XT(I−1neeT)X]=Tr[(I−1neeT)XXT] \begin{aligned} \overline X &= \frac{1}{n} e^T X \\ \sum_{i} (X_i - \overline X)^2 &= (X - \frac{1}{n} e e^T X)^T (X - \frac{1}{n} e e^T X) \\ &= X^T (I - \frac{1}{n} e e^T)^T (I - \frac{1}{n} e e^T)X \\ &= X^T (I - \frac{1}{n} e e^T)(I - \frac{1}{n} e e^T)X \\ &= X^T (I - 2 \frac{1}{n} e e^T + \frac{1}{n^2 } e e^T e e^T)X \\ &= X^T (I - 2 \frac{1}{n} e e^T + \frac{1}{n} e e^T)X \\ &= X^T (I - \frac{1}{n} e e^T)X \\ &= Tr[X^T (I - \frac{1}{n} e e^T)X] \\ &= Tr[(I - \frac{1}{n} e e^T) X X^T] \end{aligned}

Since E[XXT]=μ2eeT+σ2IE[X X^T] = \mu^2 e e^T + \sigma^2 I,

E[∑i(Xi−X‾)2]=Tr[(I−1neeT)E[XXT]]=Tr[(I−1neeT)(μ2eeT+σ2I)]=Tr[(eeT−eeT)μ2+(I−1neeT)σ2]=Tr[(I−1neeT)σ2]=Tr[I−1neeT]σ2 \begin{aligned} E[\sum_{i} (X_i - \overline X)^2] &= Tr[(I - \frac{1}{n} e e^T) E[X X^T]] \\ &=Tr[(I - \frac{1}{n} e e^T)(\mu^2 e e^T + \sigma^2 I)] \\ &=Tr[(ee^T - ee^T)\mu^2 + (I - \frac{1}{n} e e^T)\sigma^2] \\ &=Tr[(I - \frac{1}{n} e e^T)\sigma^2] \\ &=Tr[I - \frac{1}{n} e e^T] \sigma^2 \end{aligned}

where n−1n - 1 is the trace as well as the rank of I−1neeTI - \frac{1}{n}e e^T.