Conversion Optimisation
Statistical Significance
Plain English
Knowing if results are real or just luck.
Definition
Statistical significance indicates whether the results of an glossaryExperimentAn experiment is a structured test used to evaluate hypotheses and measure outcomes.Open glossary term are likely due to real effects rather than chance.
In practice
Used to validate guideA/B TestingA/B, split and multivariate testing compared: how to choose the right online experiment, run it honestly, and read the result.Open guide results before making decisions.
In context
It gets checked daily, which is how false winners are declared. Peeking at a running test until it crosses the threshold guarantees crossing it eventually.
The reality
Misunderstood or ignored, leading to false conclusions.
Compared with
Statistical Significance vs Practical Significance
Significance says the effect is probably real. Practical significance asks whether it is large enough to matter. On high glossaryTrafficTraffic refers to the number of users visiting a website, app, or digital product over a given period.Open glossary term, a statistically certain quarter-percent lift can cost more to maintain than it earns.
FAQ
Common questions
A few practical answers to the questions that usually come up around this term.
What is statistical significance?
It shows whether results are likely real and not random.
Why is it important?
It ensures decisions are based on reliable glossaryDataData is raw, uninterpreted information collected and stored so it can be analysed, processed, or used to inform decisions. On its own it carries no meaning; context and interpretation are what make it useful.Open glossary term.
How is it measured?
Using probability and glossaryConfidenceConfidence is the level of certainty in a decision or outcome based on available evidence.Open glossary term levels.
What is a common mistake?
Stopping tests too early.
Related Services
Related Guides
Related Terms
