Context is everything in AI communication

Context is all you need. Context is the only thing that matters. It's such an important piece of getting a language model to do anything for you.

My whole position on this is prompt engineering is a very human thing. When we want to get some value out of a human, we do this prompt engineering. We try to effectively communicate with that human in order to get the best output.

Crap in crap out. If you ask a pretty basic question, you're going to get a pretty basic response. And actually the same thing is true for humans, and you can think of a great example of this. When I go to another human and I ask, 'How's your day going,' they say, 'It's going pretty good.'

In this context, "prompts" refers to text instructions given to AI language models like ChatGPT to complete tasks.

Studies have shown that using bad prompts can get you down to 0% on a problem, and good prompts can boost you up to 90%.

The speakers are discussing manual analysis of AI conversation logs ("traces") to identify errors, rather than asking an LLM to automatically detect problems.

What we usually find when we try to ask an LLM to do this error analysis is it just says the trace looks good because it doesn't have the context needed to understand whether something might be bad product smell or not.