Skip to content
~/reflective_data

analytics-dictionary / term / experimentation

Sequential testing

Sequential testing uses statistical methods that remain valid when you check results repeatedly while a test runs.

Written by Reflective Data experts · updated Oct 5, 2026

Classic fixed-horizon tests require you to wait for the planned sample size. Sequential methods allow early stopping for clear winners or losers while controlling error rates.

Many modern platforms use them to avoid the peeking problem.

related

A/B testFrequentist statisticsPeeking problem

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

more in experimentation

optimizely.md

Optimizely

Optimizely is a digital experience platform known for web experimentation, feature experimentation and personalisation.
simpsons-paradox.md

Simpson’s paradox

Simpson's paradox is when a trend that appears in several groups reverses when the groups are…
type-1-and-type-2-errors.md

Type I and type II errors

A type I error is a false positive, declaring an effect that is not real; a…
minimum-detectable-effect.md

Minimum detectable effect (MDE)

The MDE is the smallest change your test is designed to reliably detect.
feature-flag.md

Feature flag

A feature flag is a switch in code that turns a feature on or off for…
multi-armed-bandit.md

Multi-armed bandit

A multi-armed bandit shifts traffic towards better-performing variants while a test is still running.

← full analytics dictionary