analytics-dictionary / term / experimentation

A/A test

An A/A test splits users into two groups that get the same experience, to check that your experimentation set-up is trustworthy.

Written by Reflective Data experts · updated Oct 5, 2026

Because there is no real difference, the test should show no significant result about 95 percent of the time. Frequent false winners point to problems in assignment, tracking or analysis.

Teams run A/A tests when adopting a new tool or after changing infrastructure.

A/B test Sample ratio mismatch (SRM)Statistical significance

// how we can help

We set up, audit and run experimentation programs on any tool. A/B testing services → · Audits

CUPED

CUPED reduces variance in experiments by using pre-experiment data about each user, so tests reach conclusions…

exposure-logging.md

Exposure logging

Exposure logging records the moment a user is actually shown a variant, so analysis only includes…

server-side-testing.md

Server-side testing

Server-side testing runs experiments in your backend, so users get the assigned variant before the page…

growthbook.md

GrowthBook

GrowthBook is an open-source experimentation and feature flagging platform that works with your own data warehouse.

kill-switch.md

Kill switch

A kill switch is a feature flag that can instantly turn a feature off in production.

guardrail-metric.md

Guardrail metric

A guardrail metric is a measure you monitor during a test to make sure the change…

← full analytics dictionary