Terms you need to know before moving on to the article: Population, Sample and Standard Deviation
Population: Population contains all the members that are present in the data
Sample: Sample contains some members of the data
Standard Deviation: Standard deviation is a measure of how spread out a dataset is.
Let’s look at the formula of standard deviation for both population and sample.
Formula for Population Standard Deviation is:
Formula for Sample Standard Deviation is:
Now let’s move on to the main part of the article.
If you have the complete data, like in the case of population, then our results will be 100% accurate and we don’t have to worry about anything.
But in most of the cases, we won’t be having the complete data, we have to make assumptions only based on some data, like in the case of a sample which is a part of the population, we can’t be sure whether we can generalize our results.
So, if we use N in the formula for sample standard deviation also, then there is a possibility that we may underestimate the value of standard deviation. That is why N-1 is used in the formula, this is referred to as the Bessel’s Correction.
For understanding this, let’s look at an example:
Let’s assume that a dataset is normally distributed. So, if you take random points from the data for calculating sample distribution, then there is a high possibility that the points that you are choosing are in the middle(near mean value — highly probable points). There is a very low chance that the sample will have points that have low probability (far from the mean).
The numerator in the standard deviation formula contains squared summation of difference from the mean value. While taking the points for the sample, as most of the points are near to mean( high probability), the residual value will be lower than that of the population.
So the standard deviation/ variance value will be lower than the real value as the points that are far away from mean will not be counted.
That is why we are using N-1 in the denominator to resolve the bias created while choosing the points.
Thanks for reading through the article. Follow us to get notified of future content like this.