A/B testing has fundamental limitations

You can't experiment your way out of everything. What people decide to test in a world that has promoted experimentation for everything tends to be more incremental by design.

I think growth and growth hacking and doing all this analytics, A/B testing stuff, is a total waste of time for very early startups.

Sometimes you don't need to experiment. Sometimes if the business, let's say activation, right, activation rates go up 6% versus 7%, that precision actually doesn't help all that much beyond being able to say in your performance review, 'Hey, I increased activation by 7%.'

A/B experiments also don't work, and that's a very unique thing about marketplaces. I mean, not work the way that you would expect them to work because there are network effects all over.

If we do an AB test, there has to be a hypothesis. If we don't have a hypothesis and A is better than B, then we're stuck with B. And that's a really, really big problem, but we can never change it.

I think that it maybe misses the point in some changes or some areas where you are working towards a bigger net new thing or this specific change won't really be indicative of the greater whole you're building towards.