Case Study: Solving The Discrepancy Between A/B Testing Tool, Google Analytics and Backend Data For a Large E-Commerce Business

Working with skewed data can be worse than having no data at all. This is why we’re always promoting all sorts of analytics audits and making sure all data sources agree with each other. At the very least, you should know why the numbers in different tools don’t match (i.e. analytics doesn’t include offline sales but backend does).

Our client in this case study contacted us with a quite specific problem. They were running a decent CRO program with 4-6 A/B experiments running every month. The problem they had with the program, though, was that the numbers they saw in their testing tool Optimizely, Google Analytics and backend didn’t match. In fact, there was a ~35% discrepancy overall.

Read more »