analytics-dictionary / term / experimentation

Variance reduction

Variance reduction techniques make experiments more sensitive, so they reach conclusions with less data.

Written by Reflective Data experts · updated Oct 5, 2026

CUPED uses pre-experiment behaviour to explain away noise; stratification balances groups on key attributes. The result is narrower confidence intervals and shorter tests.

They require good historical data and careful implementation.

CUPED Minimum detectable effect (MDE)Statistical power

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

Canary release

A canary release exposes a new version to a small group of users to detect problems…

statistical-significance.md

Statistical significance

A result is statistically significant when it is unlikely to be explained by chance alone, given…

kill-switch.md

Kill switch

A kill switch is a feature flag that can instantly turn a feature off in production.

adobe-target.md

Adobe Target

Adobe Target is Adobe testing and personalisation tool, part of Adobe Experience Cloud.

personalization.md

Personalization

Personalization tailors content or offers to individual users or segments, based on their behaviour or attributes.

aa-test.md

A/A test

An A/A test splits users into two groups that get the same experience, to check that…

← full analytics dictionary