Test everything, but test the right version

I'm very clear that I'm a big fan of test everything, which is any code change that you make, any feature that you introduce has to be in some experiment. Because again, I've observed this sort of surprising result that even small bug fixes, even small changes can sometimes have surprising, unexpected impact.

You never want to walk away from an experiment or test and say, 'Well, maybe the execution was bad because it takes a lot of energy to mobilize a team to test something,' and you really want to make sure your tests actually provide signal.

I'd say test everything, we've run in the last four years over 600 experiments on the streaks, so every other day.

Find a place, find a team where experimentation is easy to run. Don't go with the team that launches every six months, or Office used to launch every three years. Go with the team that launches frequently.

I didn't let the team run this experiment for six months so that we focus on lower hanging fruit. We ran this very simple experiment that was like, 'You sign up and then you get a badge.' And of course, in retrospect, it led to no results, because no one is proud of signing up.