101–107 of 107 posts
Earlier quoted context omitted.
do you believe everything you see in adverts?
I don’t expect ads to contain materially false statements these days, as you can get in trouble for that.
Earlier quoted context omitted.
> you are standing in front of your dishwasher waiting for it to grow arms and do your dishes in the sink. No. I'm standing in front of the dishwasher and the dishwasher expects me to tell it in detail how to wash the dishes. This is not about if you can find any use for a LLM at all. This is about: > LLMs are still surprisingly bad at some simple tasks And yes. They are bad if you have to hand feed them each and eve…
>Hand feed them every detail for an extremely simple task like comparing two lists You believe 57 words are "each and every detail", and that "produce two full, exhaustive lists of items out of your blackbox inner conceptspace/fetch those from the web" are "extremely simple tasks"? Your ignorance of how complex these problems are misleads you into believing there's nothing to it. You are trying to supply an abstracti…
Sure they are. I'm not interested in how difficult this is for a LLM. This is not the question. Go out there, get the information. That this is hard for a LLM proves the point: They are surprisingly bad at some simple tasks.
> Try learning programming.
I started programming in the early 1980's.
Earlier quoted context omitted.
>Hand feed them every detail for an extremely simple task like comparing two lists You believe 57 words are "each and every detail", and that "produce two full, exhaustive lists of items out of your blackbox inner conceptspace/fetch those from the web" are "extremely simple tasks"? Your ignorance of how complex these problems are misleads you into believing there's nothing to it. You are trying to supply an abstracti…
> You believe 57 words are "each and every detail", and that "produce two full, exhaustive lists of items out of your blackbox inner conceptspace/fetch those from the web" are "extremely simple tasks"? Sure they are. I'm not interested in how difficult this is for a LLM. This is not the question. Go out there, get the information. That this is hard for a LLM proves the point: They are surprisingly bad at some simple…
And neither was that my point. It is a complex problen, full stop. Again, your own inability to look past your personal abstractions ("just do the thing, it's literally one step dude") is what makes it feel simple. You ever do that "instruct someone to make coffee" exercise when you started out? What you're doing is saying "just make the coffee", refusing to decompose the problen any further, and then complaining that the other person is bad at following instructions.
So are people?
https://grok.com/share/bGVnYWN5_25cf6423-c1e0-457f-99a0-570f...
> I think it comes down to how familiar you are with the domain and its constraints. When I watch a medical drama, I have no idea if they're using realistic language. It sounds good, but real doctors probably cringe at the inaccuracies. By now, numerous notable programmers have reported positive experiences with all forms of AI-assisted coding, which this conclusion arrogantly fails to account for.
Earlier quoted context omitted.
> Or, if LLMs are so smart, why doesn't it say "Hmmm, would you like to use a different model for this?" That is indeed an area where LLMs don't shine. That is, not only are they trained to always respond with an answer, they have no ability to accurately tell how confident they are in that answer. So you can't just filter out low confidence answers.
Something I think would be interesting for model APIs and consumer apps to exposed would be the probability of each individual token generated. I’m presuming that one class of junk/low quality output is when the model doesn’t have high probability next tokens and works with whatever poor options it has. Maybe low probability tokens that cross some threshold could have a visual treatment to give feedback the same way…
Isn't that what logprobs is?