Definition
The significance level α is the maximum risk of a Type I error — rejecting a null hypothesis that is actually true (a false alarm). A Type II error (β) is the opposite: the effect exists but the test misses it. The usual choice is α = 0.05; the power 1 − β says how reliably you detect a real effect.
Foundations & hypothesis testing
The two errors pull against each other: a stricter α means fewer false alarms but more missed real effects. The only way to reduce both is a larger sample or more precise measurement — which is why sample size is planned in advance for a target power (typically 80 %).
With many tests at once (dozens of variables, several groups) the risk of a false alarm grows quickly — with 20 tests at α = 0.05 the chance of at least one falsely significant result is about 64 %. The remedies are corrections (Bonferroni, Holm, FDR) or post-hoc tests that build them in.
In Statistica
Set the significance level for highlighting results in each module; ANOVA and post-hoc tests in Statistica offer multiple-comparison corrections directly (Tukey, Scheffé, Bonferroni). Planning sample size from α and power is handled by the Power Analysis module.
Related terms
- p-valueThe p-value is the probability of obtaining a result at least as extreme as the one observed if the null hypothesis…
- Statistical power and sample sizeStatistical power is the probability that a test detects an effect that really exists (1 − β).
- Post-hoc testsPost-hoc tests follow a significant ANOVA and compare groups pairwise to show which ones really differ.
- Effect sizeEffect size expresses how large a difference is or how strong an association is — independently of sample size.
Knowledgebase guides
FAQ
- Why 0.05?
- It is a historical convention (R. A. Fisher). In practice choose α by the cost of an error — 0.10 may be fine for screening, 0.01 for a confirmatory study.
- What is FDR?
- False discovery rate — the expected share of false positives among significant results. It is used with hundreds of tests (genomics, data mining) instead of the overly strict Bonferroni.
Try it on your own data
Statistica free for 30 days
Full version, no credit card. Or get a pricing estimate in a minute.
Foundations & hypothesis testing
Updated: September 2026.