analytics-dictionary / term / experimentation

Confidence interval

A confidence interval is a range of values that likely contains the true effect, given the data.

Written by Reflective Data experts · updated Oct 5, 2026

A 95 percent interval around a lift of 3 percent might run from 0.5 to 5.5 percent. It shows both the estimate and the uncertainty around it, which is more useful than a single number.

If the interval includes zero, you cannot rule out that there is no effect.

Frequentist statistics P-value Statistical significance

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

Win rate

Win rate is the percentage of experiments that produce a significant improvement.

variance-reduction.md

Variance reduction

Variance reduction techniques make experiments more sensitive, so they reach conclusions with less data.

ab-tasty.md

AB Tasty

AB Tasty is an experimentation and personalisation platform for web and product teams.

fooc.md

Flash of original content (FOOC)

FOOC is when visitors briefly see the original page before a client-side test applies the variant.

personalization.md

Personalization

Personalization tailors content or offers to individual users or segments, based on their behaviour or attributes.

experiment-velocity.md

Experiment velocity

Experiment velocity is how many experiments a team completes in a period.

← full analytics dictionary