It is true that they don't tend to give simple answers.
But it reveals something about human biases that we tend to seek out simple answers to complex questions. We tend to think that because a question is short, that it's simple. We forgot how many details are involved and how much is actually going on underneath. Reality itself is often annoyingly complex.
All living creatures, including humans, are conditioned to simplify extremely complex things into simple concepts which we can react to. We only understand things to the extent that we believe they affect us.
Far easier to simply spread peanut butter and jelly on bread than to consider how the peanuts were harvested, how many bugs were in the fruits, how close the wheat was to molding, how much sugar is in the bread, how many years it tooks us to perfect such tasty and long lasting ingredients, what potenitally substandard conditions the labor involved endured...etc, every time one wants a sandwhich.
And then a multiple of that easier as well in conversing about a peanut butter and jelly sandwhich as few if any of the parties will think about any of the above.
We see the same in computers. A web dev can make it quite far without once considering the OSI model. A game dev can complete an entire career without ever worrying about the underlying bytecode.
There are situational benefits to diving deeper, of course, but I don't imagine we'd make it very far if we had to reinvent the universe for every word in every conversation or chain of thought.
Every time one of these consumer data brokers/gatherers shows some insight into how they gather the data it makes me feel sick. I know they're doing it, I know it's granular, it's still gross.
Completely unrelated to the article itself but I landed on the posthog.com homepage after viewing their open positions and the landing page is cool. It's sort of a desktop UI and different from what I usually see, was expecting some other HR domain to come up.
Contrarian take.
I think everyone should use LLMs and the most expensive possible, unless they know how to run evals (both on line and off line).
Yes its expensive, ugly, etc.
But...
I've seen enough IRL that people just create really bad evals that do not reflect real world. Then the product is just bad. Outright broken for the p90 customer.
People don't know statistics. Offline Evals are hard. Sampling the entire population distribution is hard. Sampling the fat tail is harder. Online evals are big ops projects that are giant balls of death.
Maybe I should write up a post about this, but about evals ... I have feelings.
Yes you're right that writing evals is hard. And it comes down to the simple fact that the thing that you're testing - a LLM-driven feature that presumably suffers from variance and inadequate accuracy - and your measuring device - an LLM judge - suffer from the fundamental problems. But the lure of natural language is such that instead of seeing this pipeline of LLM-driven things as a path of cascading errors, it is easy to view it as one set of problems correcting for another set.
In this world, the LLM-driven feature is a probabilistic entity, and the judge layered on top of that is another probabilistic entity. The real work is to look at the judge's response and infer the performance of the feature. Good evals make the inference step "pass-through", i.e., the judge outcomes have a 1:1 mapping with feature outcomes. But this kind of reliability often doesn't naturally appear. I think of reliability as being composed of three attributes: (a) consistency: do we get the same result for an eval run on the same llm with the same input (b) inter-llm agreement: do outcomes agree over multiple trials of running an eval over multiple llms with the same input (c) accuracy: does the eval agree with human judgement. You need extra work to guarantee these properties. And even when you can, you won't get 100% reliability, but say high 90s, and you still to make a judgement call as to call a feature "functional".
Sure it's an LLM, it's just neither generative nor instruction-tuned, like BERT, which gave them a reason to market it as not being one. Though modern "LLMs" are also modeling a lot more than just the language so they're kind of hard to define. Not much point in being pedantic, let them call anything anything and see what sticks.
Related: Amazon saves millions by replacing Sonnet 4.6 with Strands Decider - https://news.ycombinator.com/item?id=50436640
It is true that they don't tend to give simple answers.
But it reveals something about human biases that we tend to seek out simple answers to complex questions. We tend to think that because a question is short, that it's simple. We forgot how many details are involved and how much is actually going on underneath. Reality itself is often annoyingly complex.
All living creatures, including humans, are conditioned to simplify extremely complex things into simple concepts which we can react to. We only understand things to the extent that we believe they affect us.
That's just a caloric efficiency thing, isn't it?
Far easier to simply spread peanut butter and jelly on bread than to consider how the peanuts were harvested, how many bugs were in the fruits, how close the wheat was to molding, how much sugar is in the bread, how many years it tooks us to perfect such tasty and long lasting ingredients, what potenitally substandard conditions the labor involved endured...etc, every time one wants a sandwhich.
And then a multiple of that easier as well in conversing about a peanut butter and jelly sandwhich as few if any of the parties will think about any of the above.
We see the same in computers. A web dev can make it quite far without once considering the OSI model. A game dev can complete an entire career without ever worrying about the underlying bytecode.
There are situational benefits to diving deeper, of course, but I don't imagine we'd make it very far if we had to reinvent the universe for every word in every conversation or chain of thought.
PostHog should not be using them on customer data, period.
Every time one of these consumer data brokers/gatherers shows some insight into how they gather the data it makes me feel sick. I know they're doing it, I know it's granular, it's still gross.
Gathering it and pumping it all through Anthropic and OpenAI data collectors.
I mean look at what they sell. Surveillanceware.
I have been using structured json outputs since gpt 4 days. Never understood this jev hype
i think the main deal is speed right?
Completely unrelated to the article itself but I landed on the posthog.com homepage after viewing their open positions and the landing page is cool. It's sort of a desktop UI and different from what I usually see, was expecting some other HR domain to come up.
Contrarian take. I think everyone should use LLMs and the most expensive possible, unless they know how to run evals (both on line and off line).
Yes its expensive, ugly, etc. But...
I've seen enough IRL that people just create really bad evals that do not reflect real world. Then the product is just bad. Outright broken for the p90 customer.
People don't know statistics. Offline Evals are hard. Sampling the entire population distribution is hard. Sampling the fat tail is harder. Online evals are big ops projects that are giant balls of death.
Maybe I should write up a post about this, but about evals ... I have feelings. Yes you're right that writing evals is hard. And it comes down to the simple fact that the thing that you're testing - a LLM-driven feature that presumably suffers from variance and inadequate accuracy - and your measuring device - an LLM judge - suffer from the fundamental problems. But the lure of natural language is such that instead of seeing this pipeline of LLM-driven things as a path of cascading errors, it is easy to view it as one set of problems correcting for another set.
In this world, the LLM-driven feature is a probabilistic entity, and the judge layered on top of that is another probabilistic entity. The real work is to look at the judge's response and infer the performance of the feature. Good evals make the inference step "pass-through", i.e., the judge outcomes have a 1:1 mapping with feature outcomes. But this kind of reliability often doesn't naturally appear. I think of reliability as being composed of three attributes: (a) consistency: do we get the same result for an eval run on the same llm with the same input (b) inter-llm agreement: do outcomes agree over multiple trials of running an eval over multiple llms with the same input (c) accuracy: does the eval agree with human judgement. You need extra work to guarantee these properties. And even when you can, you won't get 100% reliability, but say high 90s, and you still to make a judgement call as to call a feature "functional".
Can you elaborate on what you mean by Eval? All I can think of is eval() in js...
I hope you agree people should use the "right tool for the job" meaning a calculator for arithmetic rather than an LLM for everything.
The approach of using decision models to gate more expensive work.. I mean, yeah.
But using protecting PII as an example use case is a bit bonkers?
"Not everything needs an LLM! Use a decision model!"
And what is a decision model exactly? An LLM with instructions to limit responses?
Huh.
Sure it's an LLM, it's just neither generative nor instruction-tuned, like BERT, which gave them a reason to market it as not being one. Though modern "LLMs" are also modeling a lot more than just the language so they're kind of hard to define. Not much point in being pedantic, let them call anything anything and see what sticks.
Sounds like a yes/no question.
tl;dr Not everything needs an LLM, but everything needs a typesafe wrapper around an LLM.
Gotcha
Ridiculous field we are in, people seemingly have forgotten the absolute basics in under a year.