Statistical Test Calculator

Choosing the test is the part that gets marked wrong. Work down three questions: what type is the response, how many groups, and are the groups paired?

Situation	Test	Statistic
One mean, $\sigma$ known	one-sample $z$-test	$z = \dfrac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}$
One mean, $\sigma$ unknown	one-sample $t$-test, $df = n-1$	$t = \dfrac{\bar{x} - \mu_0}{s/\sqrt{n}}$
One proportion	one-proportion $z$-test	$z = \dfrac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}}$
Two independent means	two-sample (Welch) $t$-test	$t = \dfrac{\bar{x}_1 - \bar{x}_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}}$
Paired measurements	paired $t$-test on the differences	$t = \dfrac{\bar{d}}{s_d/\sqrt{n}}$
Counts in categories	chi-square test	$\chi^2 = \sum \dfrac{(O-E)^2}{E}$

Note the proportion test uses $p_0$, the null value, in the standard error — not $\hat{p}$. That differs from the interval formula, which uses $\hat{p}$.

The five steps

1. State $H_0$ and $H_a$, and whether $H_a$ is one- or two-sided.
2. Check conditions (below) and fix $\alpha$, usually $0.05$.
3. Compute the test statistic from the table above.
4. Compare it to the critical value, or compare the $p$-value to $\alpha$.
5. Conclude in context — "reject $H_0$", never "prove $H_a$".

Critical values you will reuse

Two-sided $\alpha$	$z^*$	$t^*_{19}$	$t^*_{24}$
0.10	1.645	1.729	1.711
0.05	1.96	2.093	2.064
0.01	2.576	2.861	2.797

The matching interval

Every two-sided test has a companion interval: estimate $\pm$ (critical value) $\times$ (standard error). A two-sided test at $\alpha$ rejects $H_0$ exactly when the $(1-\alpha)$ interval excludes $\mu_0$, so the two approaches never disagree. The interval's endpoints are the confidence limits.

Every test above assumes:

• Random sampling or random assignment. Without it the sampling distribution is undefined and no $p$-value means anything.
• Independent observations, with $n \le 10\%$ of the population when sampling without replacement. Paired data violate independence between groups on purpose — that is why the paired test analyses the differences.
• Approximate normality: the population is roughly normal, or $n \ge 30$ so the Central Limit Theorem applies. For a proportion, check $np_0 \ge 10$ and $n(1-p_0) \ge 10$. Chi-square needs every expected count $\ge 5$.

Frequent errors

• Using $z$ because $n$ "feels" big while $\sigma$ is estimated by $s$ — the $t$-test is the correct choice regardless of $n$.
• Running a two-sample test on paired data, which throws away the pairing and inflates the standard error.
• Halving or doubling the $p$-value in the wrong direction between one- and two-sided tests.
• Reporting "fail to reject" as "$H_0$ is true". It only means the evidence was insufficient.