Why experts writing AI evals is creating the fastest-growing companies in history

"Kodak" appears to be a transcription error for "Claude," which along with ChatGPT and Cursor are AI coding tools used for software development.

I think the people that can leverage AI to do whatever their day-to-day workflows are... We'll give people interviews where we say, 'Use ChatGPT and Kodak. Use Claude code. Use whatever tool cursor and whatever tools are available to build a website and let's see what product you're able to build in an hour.'

It's some combination of being stubborn with respect to your thesis around how the world will change, but also very open-minded with respect to exactly what form that takes and how the market's developing.

In AI development, "evals" refers to evaluation benchmarks used to test and measure AI model performance on specific tasks.

If the model is the product, then the eval is the product requirement document.

"Sheer" appears to be a transcription error for "steer" - referring to steering/guiding business direction based on market pull/demand signals.

Sometimes you just need to sheer it from the market and know that it's there, the poll, to know the right places to focus. Because if it's difficult to sell, if it's extremely difficult to sell the marginal customer, you're not going to be able to grow a huge business.

You don't want to fight against them using the models. It's similar to when the calculator came out, you don't want to give people all of this arithmetic work of how do you get them to do it and not use the calculator. You want to tell them, 'Use the tools and let's see what you can do.'

Foody is CEO of Mercor, which helps AI companies hire experts to evaluate and improve AI model performance.

The wealthiest companies in the world are willing to spend whatever it takes to improve model capabilities.

Models are only as good as their evals.