analytics-dictionary / term / experimentation

Effect size

Effect size is the magnitude of a difference between variants, not just whether it exists.

Written by Reflective Data experts · updated Oct 5, 2026

A result can be statistically significant and still too small to matter. Reporting effect size with a confidence interval tells decision makers how much to expect.

Express it in business terms, such as additional orders per thousand visitors.

Confidence interval Minimum detectable effect (MDE)Statistical significance

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

Sequential testing

Sequential testing uses statistical methods that remain valid when you check results repeatedly while a test…

one-tailed-vs-two-tailed-test.md

One-tailed vs two-tailed test

A one-tailed test looks for an effect in one direction only; a two-tailed test looks for…

experiment-velocity.md

Experiment velocity

Experiment velocity is how many experiments a team completes in a period.

exposure-logging.md

Exposure logging

Exposure logging records the moment a user is actually shown a variant, so analysis only includes…

practical-significance.md

Practical significance

Practical significance asks whether a measured effect is large enough to matter to the business.

multiple-comparisons.md

Multiple comparisons problem

The multiple comparisons problem is that testing many metrics or variants raises the chance of at…

← full analytics dictionary