Statistics Essentials

This is the 'how it works' post, and the cover frames the entire field around three questions: where is the middle, how spread out is the data, and what is its shape. That triad is genuinely most of descriptive statistics, and organizing your thinking around it makes the formulas feel purposeful rather than arbitrary.

We then add one near-magical result, the central limit theorem, which is the bridge from describing data to making inferences about it. Together these give you the mechanical core of the discipline.

Measures of center each answer 'what is a typical value,' but they disagree in revealing ways. The mean is the arithmetic average and uses every value, which makes it efficient but fragile — a single huge outlier drags it far from the bulk of the data. The median is the middle value when sorted, so it ignores how extreme the extremes are and stays robust.

The mode is the most frequent value, most useful for categorical or multimodal data. The practical rule: when a distribution is skewed, the median usually describes 'typical' more honestly than the mean, which is why you see median income and median home price in serious reporting.

Spread tells you how much values vary around the center, and it's just as important as the center itself. Variance is the average of the squared distances from the mean. Squaring keeps everything positive and punishes large deviations more heavily, but it leaves the units squared (dollars-squared), which is hard to interpret.

Standard deviation is the square root of variance, bringing the number back into the original units. A small standard deviation means the data clusters tightly around the mean; a large one means it's scattered. Two datasets can share an identical mean and behave completely differently, and only the spread reveals that.

This code slide computes variance and standard deviation by hand so the formula isn't a black box. Notice the division by (x.size − 1) rather than x.size — this is the sample variance, using Bessel's correction. The mean is subtracted from each value, the differences are squared, summed, and divided.

Doing it manually once demystifies what np.std(ddof=1) does internally. The mean here is 5.0, the sample variance about 4.57, and the standard deviation about 2.14, meaning typical values sit roughly 2 units away from the center.

The n−1 question trips up almost everyone, and the intuition is worth holding onto. When you estimate the population variance from a sample, you measure deviations from the sample mean — but the sample mean is itself fitted to that same data, so it sits closer to the data than the true population mean does. This makes the squared deviations systematically too small.

Dividing by n−1 instead of n inflates the estimate just enough to correct this bias on average. The n−1 is called the degrees of freedom: one is 'used up' estimating the mean. This is exactly why NumPy needs ddof=1 to give you the unbiased sample variance.

Shape is the third question, and skew is the most consequential aspect of it. A right-skewed distribution has a long tail stretching toward high values, which pulls the mean above the median — incomes and house prices are the classic examples. A symmetric distribution, like the normal, has the mean and median sitting on top of each other.

Knowing the shape tells you which center to trust and whether transformations might help. A right-skewed target often becomes well-behaved after a log transform, which is why checking shape before modeling saves so much trial and error.

The normal distribution is the most important shape in statistics, fully described by just two numbers: its mean μ (where it's centered) and its standard deviation σ (how wide it is). Its defining gift is the 68-95-99.7 rule: about 68% of values fall within one σ of the mean, 95% within two, and 99.7% within three.

This rule is the fastest gut-check in all of statistics. If a value is more than two standard deviations from the mean, it's in the outer 5% — unusual. More than three, and it's genuinely rare. You can eyeball whether an observation is surprising without any further computation.

The central limit theorem is the result that makes the normal distribution unavoidable, and it's almost magical. Take any distribution — skewed, lumpy, weird — draw a sample, and compute its mean. Repeat many times. The distribution of those sample means approaches a normal distribution as the sample size grows, no matter what the original data looked like.

This is why so much inference assumes normality even when raw data isn't normal: we're usually reasoning about means and other averages, and the CLT guarantees those are approximately normal. It's the theoretical backbone of confidence intervals and t-tests on real, messy data.

This code demonstrates the central limit theorem empirically, which is far more convincing than the proof. The source data is exponential — strongly right-skewed, nothing like a bell curve. We draw 10,000 samples of size 30 and take each sample's mean. Those means cluster into a tidy normal distribution centered on the true mean of 2.

Two things to notice: the average of the sample means recovers the population mean, and the spread of the sample means shrinks roughly as 1/√n. That shrinking is exactly why larger samples give more precise estimates, and it's the engine behind the standard error formula.

A confidence interval converts a single point estimate into an honest range, and its definition is more subtle than people assume. A 95% confidence interval is a range constructed so that, if you repeated the whole sampling process many times, about 95% of the intervals you build would contain the true parameter.

Mechanically it's the estimate plus or minus a margin. That margin grows with the data's spread (more variability, less certainty) and shrinks with sample size (more data, more certainty), scaling with σ/√n. The interval is the practical, everyday output of inferential statistics — it tells your reader not just your guess but how much to trust it.

This flow diagram captures the whole move from estimate to interval in three steps. You start with the sample mean as your point estimate. You add and subtract a margin equal to z times the standard error σ/√n. The result is a 95% interval — an honest range rather than a falsely precise single number.

The takeaway is that the interval is not an afterthought; it's the proper form of a statistical answer. A point estimate without an interval discards the information about how reliable the estimate is, which is often the most decision-relevant part.

The closing mistake — reporting a mean for skewed data — is one of the most common and most misleading errors in practice. Quoting an average salary of $95k when one founder earns $5M and everyone else earns $50k is technically correct arithmetic and a borderline lie about what's typical.

For skewed distributions, lead with the median, which resists the outlier, and show the spread or a few quantiles so the reader sees the shape. The mean still has its place, but presenting it alone for skewed data manufactures a misleading impression of the typical case.

That covers the mechanics: center, spread, shape, and the central limit theorem that ties descriptive summaries to inferential claims. With these in hand, you can read any one-dimensional dataset and know which summary to trust.

The next post puts all of it into runnable code — describing a real dataset, running a significance test, and building a confidence interval, so the formulas become something you've actually executed rather than just read.