Skip to main content
AmigoFish

1046: Weighing a Feather: How to Design A/B Tests That Work, with Billur Engin

From Super Data Science: ML & AI Podcast with Jon Krohn · · 24 min

This plays from the publisher's own server. We do not host a copy.

Open at the source

Not rated yet

Description

In Episode #1046, Billur Engin (who leads data science and analytics at Prolific) joins Jon Krohn to explain how to design experiments that actually teach you something. Too many A/B tests, she says, are like bathroom scales trying to weigh a feather the effect is real, but the system was never built to detect it. In her first-ever podcast appearance, Billur walks through why complexity has to earn its place in your models, the biggest myth about A/B testing, the AA test she runs to catch broken measurement, three ways to get at causality when you can’t run a controlled experiment, and why causal inference matters more than ever now that prediction has become cheap.

Interested in sponsoring a SuperDataScience Podcast episode? Email natalie@superdatascience.com for sponsorship information.⁠⁠⁠

In this episode you will learn:

-

- (01:18) What Prolific does and why representative data matters

-

- (05:19) Why complexity has to earn its place

-

- (07:29) The biggest myth about A/B testing

-

- (11:34) Why causal inference matters more than ever

-

- (14:39) Getting at causality when you can’t run an experiment

Back to Super Data Science: ML & AI Podcast with Jon Krohn · Report a problem