Skip to content
~/reflective_data

analytics-dictionary / term / experimentation

Practical significance

Practical significance asks whether a measured effect is large enough to matter to the business.

Written by Reflective Data experts · updated Oct 5, 2026

With enough traffic, tiny differences become statistically significant. Deciding the smallest worthwhile change in advance prevents celebrating results that do not move the needle.

It ties directly to the minimum detectable effect.

related

Effect sizeMinimum detectable effect (MDE)Statistical significance

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

more in experimentation

ab-test.md

A/B test

An A/B test compares two versions of something by showing each to a random group of…
eppo.md

Eppo

Eppo is an experimentation platform built to run directly on your data warehouse.
holdout-group.md

Holdout group

A holdout group is a set of users deliberately kept from a change or campaign, to…
vwo.md

VWO

VWO is an A/B testing and conversion optimisation platform with visual editing, heatmaps and surveys.
progressive-delivery.md

Progressive delivery

Progressive delivery releases a change to a small share of users first and expands gradually while…
cuped.md

CUPED

CUPED reduces variance in experiments by using pre-experiment data about each user, so tests reach conclusions…

← full analytics dictionary