1046: Weighing a Feather: How to Design A/B Tests That Work, with Billur Engin
From Super Data Science: ML & AI Podcast with Jon Krohn · · 24 min
This plays from the publisher's own server. We do not host a copy.
Description
In Episode #1046, Billur Engin (who leads data science and analytics at Prolific) joins Jon Krohn to explain how to design experiments that actually teach you something. Too many A/B tests, she says, are like bathroom scales trying to weigh a feather the effect is real, but the system was never built to detect it. In her first-ever podcast appearance, Billur walks through why complexity has to earn its place in your models, the biggest myth about A/B testing, the AA test she runs to catch broken measurement, three ways to get at causality when you can’t run a controlled experiment, and why causal inference matters more than ever now that prediction has become cheap.
Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.
In this episode you will learn:
-
- (01:18) What Prolific does and why representative data matters
-
- (05:19) Why complexity has to earn its place
-
- (07:29) The biggest myth about A/B testing
-
- (11:34) Why causal inference matters more than ever
-
- (14:39) Getting at causality when you can’t run an experiment
Back to Super Data Science: ML & AI Podcast with Jon Krohn · Report a problem