Most experiments fail - and that's exactly the point

Context: Kohavi is referring to A/B testing experiments at tech companies, where "failure" means the experiment didn't improve the target metric.

At Microsoft, about 66%, two thirds of ideas fail. At Bing, which is a much more optimized domain after we've been optimizing it for a while, the failure rate was around 85%. And then at Airbnb, this 92% number is the highest failure rate that I've observed.

If you're doing growth, right, if you're doing product-led growth the right way, then you're trying to balance the science and sort of taking a somewhat hygienic approach to validating assumptions and hypotheses with being really ambitious and really pushing for the things that are going to have massive impact for your customers at the end of the day. And when you're doing that, you're going to fail more than you're going to be successful along the way.

Growth experiments in my history are typically 30%-ish success rate. So the vast majority of things that you try don't work. And so you want to create a culture where people aren't afraid to take risks and aren't afraid to fail. And for me, failure is not that you didn't drive revenue, failure is not learning.

Roughly 80% of the times, our hypotheses and the things that we believe will be true are wrong.

If your sort of primary modality of product-led growth work is experiment-driven product development and you're hitting more than like 30, 40% of the time, you're thinking too small.

If you're not resilient, what I've seen happen is you end up sort of grasping for a win, which can sometimes look like making bets that are too small and too insignificant to matter.