P-Value Calculator

A p-value is the probability of observing test results as extreme as, or more extreme than, the actual results — assuming the null hypothesis $H_0$ is true.

Formally, for a test statistic $T$ with observed value $t$:

• Right-tailed: $p = P(T \geq t \mid H_0)$
• Left-tailed: $p = P(T \leq t \mid H_0)$
• Two-tailed: $p = 2 \cdot P(T \geq |t| \mid H_0)$

Interpretation: a small p-value means the observed data would be surprising if $H_0$ were true, so we have evidence against $H_0$. A large p-value means the data are consistent with $H_0$ — but does not prove $H_0$ is true.

Decision rule: compare $p$ to a pre-chosen significance level $\alpha$ (typically 0.05):

• $p < \alpha$ → reject $H_0$ ('statistically significant')
• $p \geq \alpha$ → fail to reject $H_0$ (not enough evidence)

What p-value is NOT:

• It is not the probability that $H_0$ is true.
• It is not the probability that the alternative $H_1$ is true.
• It is not a measure of effect size.
• It does not distinguish 'practical significance' from 'statistical significance'.

Step-by-Step

1. State the hypotheses $H_0$ and $H_1$.
2. Choose a test appropriate for the data (z-test, t-test, chi-square, F-test, ...).
3. Compute the test statistic from the data.
4. Determine the tail(s) based on $H_1$: right-tailed ($>$), left-tailed ($<$), or two-tailed ($\neq$).
5. Find the p-value from the test's distribution.
6. Compare to $\alpha$ and conclude.

P-Values from a Z-Statistic

For a standard normal $Z$:

• Right-tailed: $p = 1 - \Phi(z)$
• Left-tailed: $p = \Phi(z)$
• Two-tailed: $p = 2(1 - \Phi(|z|))$

Quick reference: $z = 1.96$ → two-tailed $p \approx 0.05$. $z = 2.576$ → two-tailed $p \approx 0.01$.

P-Values from a T-Statistic

Use the t-distribution with $n - 1$ degrees of freedom (or as specified by the test). Same tail logic as z, but the distribution has slightly heavier tails for small df.

P-Values from a Chi-Square Statistic

Chi-square tests are inherently right-tailed because $\chi^2 \geq 0$ and larger values indicate worse fit to $H_0$:

$p = P(\chi^2_{df} \geq \text{observed})$

One-Tailed vs Two-Tailed: Which to Use?

• Two-tailed: when you care about deviation from $H_0$ in either direction. Default in most academic settings.
• One-tailed: when the alternative hypothesis is directional and pre-specified ($H_1: \mu > 0$, not $\mu \neq 0$). Halves the p-value if the direction matches.

Never pick the tail after seeing the data — that's p-hacking.

Common Significance Thresholds

The American Statistical Association has warned against treating $\alpha = 0.05$ as a bright line — context and effect size matter more than crossing a threshold.

• 'P-value is the probability $H_0$ is true': WRONG. P-value is computed assuming $H_0$ is true; it does not measure how likely $H_0$ is.
• Treating $p = 0.049$ and $p = 0.051$ as fundamentally different: they aren't. The 0.05 threshold is a convention, not a phase transition.
• Picking the tail after seeing the data: if you see $z = -2$ and switch to a left-tailed test, you've doubled your false-positive rate. Pre-specify.
• Confusing significance with effect size: a tiny effect with a huge sample can be 'highly significant' yet practically irrelevant. Always report effect sizes alongside p-values.
• Multiple comparisons inflation: running 20 tests at $\alpha = 0.05$, one false positive is expected by chance. Use Bonferroni or FDR corrections.
• '$p > 0.05$ proves $H_0$': NO. Failing to reject is not the same as accepting. It just means the data don't have enough evidence against $H_0$ at this sample size.