41–50 of 107 posts
By now, numerous notable programmers have reported positive experiences with all forms of AI-assisted coding, which this conclusion arrogantly fails to account for.
>This is a pretty simple question to answer. Take two lists and compare them. This continues a pattern as old as home computing: The author does not understand the task themselves, consequently "holds the computer wrong", and then blames the machine. No "lists" were being compared. The LLM does not have a "list of TLDs" in its memory that it just refers to when you ask it. If you haven't grokked this very fundamental…
How would you solve that problem? You'd probably go to the internet, get the list of TLDs and the list of HTML5-Element and than compare those lists.
The author compares three commercial large‑language models that have direct internet access, but none of them appear capable of performing this seemingly simple task. I think his conclusion is valid.
They are very good at some tasks and terrible at others. I use LLMs for language-related work (translations, grammatical explanations etc) and they are top notch in that as long as you do not ask for references to particular grammar rules. In that case they will invent non-existent references. They are also good for tutor personas: give me jj/git/emacs commands for this situation. But they are bad in other cases. I s…
I think Gemini is one of the best example of an LLM that is in some cases the best and in some cases truly the worst. I once asked it to read a postcard written by my late grandfather in Polish, as I was struggling to decipher it. It incorrectly identified the text as Romanian and kept insisting on that, even after I corrected it: "I understand you are insistent that the language is Polish. However, I have carefully…
I once had Claude tell me to never talk to it again after it got upset when I kept giving it peer reviewed papers explaining why it was wrong. I must have hit the tumbler dataset since I was told I was sealioning it, which took me back a while.
Earlier quoted context omitted.
> When LLMs say something true, it’s a coincidence of the training data that the statement of fact is also a likely sequence of words; Do you know what a "coincidence" actually is? The definition you're using is wrong. It's not a coincidence that I train a model on healthcare regulations and it answers a question about healthcare regulations correctly. None of that is coincidental. If I trained it on healthcare regul…
> It's not a coincidence that I train a model on healthcare regulations and it answers a question about healthcare regulations If you train a model on only healthcare regulations it wont answer questions about healthcare regulation, it will produce text that looks like healthcare regulations.
Earlier quoted context omitted.
OP here. I literally opened up Gemini and used the defaults. If the defaults are shit, maybe don't offer them as the default? Or, if LLMs are so smart, why doesn't it say "Hmmm, would you like to use a different model for this?" Either way, disappointing.
> Or, if LLMs are so smart, why doesn't it say "Hmmm, would you like to use a different model for this?" That is indeed an area where LLMs don't shine. That is, not only are they trained to always respond with an answer, they have no ability to accurately tell how confident they are in that answer. So you can't just filter out low confidence answers.
I’m presuming that one class of junk/low quality output is when the model doesn’t have high probability next tokens and works with whatever poor options it has.
Maybe low probability tokens that cross some threshold could have a visual treatment to give feedback the same way word processors give feedback in a spelling or grammatical error.
But maybe I’m making a mistake thinking that token probability is related to the accuracy of output?
>This is a pretty simple question to answer. Take two lists and compare them. This continues a pattern as old as home computing: The author does not understand the task themselves, consequently "holds the computer wrong", and then blames the machine. No "lists" were being compared. The LLM does not have a "list of TLDs" in its memory that it just refers to when you ask it. If you haven't grokked this very fundamental…
That’s the point the author is making. The LLMs don’t have the raw correct information required to accomplish the task so all they can do is provide a plausible sounding answer. And even if it did the way they are architected still can only results in a plausible sounding answer.
>This is a pretty simple question to answer. Take two lists and compare them. This continues a pattern as old as home computing: The author does not understand the task themselves, consequently "holds the computer wrong", and then blames the machine. No "lists" were being compared. The LLM does not have a "list of TLDs" in its memory that it just refers to when you ask it. If you haven't grokked this very fundamental…
That’s the point the author is making. The LLMs don’t have the raw correct information required to accomplish the task so all they can do is provide a plausible sounding answer. And even if it did the way they are architected still can only results in a plausible sounding answer.
Ask a stupid question, get a stupid answer.
The point from the end of the post that AI produces output that sounds correct is exactly what I try to emphasize to friends and family when explaining appropriate uses of LLMs. AI is great at tasks where sounding correct is the essence of the task (for example "change the style of this text"). Not so great when details matter and sounding correct isn't enough, which is what the author here seems to have rediscovered…
> When LLMs say something true, it’s a coincidence of the training data that the statement of fact is also a likely sequence of words; Do you know what a "coincidence" actually is? The definition you're using is wrong. It's not a coincidence that I train a model on healthcare regulations and it answers a question about healthcare regulations correctly. None of that is coincidental. If I trained it on healthcare regul…
As a ChatGPT user I would have reached for the thinking model for such questions. I understand if the “auto” model doesn’t pick the right model here - but confident claims from the author should be backed up by at least this much.
> Answering the question was a little tedious and subject to my tired human eyes making no mistakes Who would do this manually? Concatenate the two lists and sort them. Use "uniq -c" to count the duplicate lines and grep to pull out the lines which occur twice. It would take a few seconds.