AI's limits reveal what makes us human

Today, you take a model and run it through a video of a couple of office rooms and ask the model to count the number of chairs. And this is something a toddler could do or maybe an elementary school kid could do, and AI could not do that.

"Evals" refers to evaluation systems used to test and measure AI product performance. The "it" refers to using AI to automatically evaluate AI systems.

The top one is, 'We live in the age of AI. Can't the AI just eval it?' But it doesn't work.

Data integration is massively painful inside organizations. It's actually impossible to even now to get access to a lot of your own internal data that you need to do your job.

I think my students and I conjectured that a very critically-overlooked ingredient of bringing AI to life is big data.