Frequentist statistics is a type of statistical inference that draws conclusions from sample data by emphasizing the frequency or proportion of the data. An alternative name is frequentist inference.
In digital analytics, Frequentist statistics is often used in A/B testing to measure if one variation is significantly better than the other.
Besides its use in digital analytics, Frequentist statistics is widely used in both academia and science. Due to its great popularity, it is easy to find guides and tutorials for both beginners and advanced users.
In the broadest sense, A/B testing is detecting which variation of your website has the highest probability of performing the best based on set criteria (actually it’s a bit more complicated but more on that later in the post). We can determine four types of probability.
- Long-term frequencies
- Physical tendencies/propensities
- Degrees of belief
- Degrees of logical support
Frequentist inference is based on the first definition, Bayesian (another popular approach to calculating probability), on the other hand, is rooted in definitions 3 and 4.
Therefore, based on the frequentist definition of probability, only repeatable random events (like the flipping of a coin) have probabilities. Furthermore, these probabilities are equal to the long-term frequency of occurrence of the events in question. It is important to understand that Frequentists don’t attach probabilities to hypotheses or to any fixed but unknown values in general. Ignoring this fact is what often leads to misinterpretations of frequentist analyses.
More on statistics in A/B testing.
Last modified: June 12, 2019
Want to see more articels like What is Frequentist Statistics? Check out all definitions in the Analytics Dictionary.