301–310 of 422 posts
Earlier quoted context omitted.
As Fowler himself states, there's a need to learn to use these tools properly. In any case poor work quality is a failure of tech leadership and culture, it's not AI's fault.
It’s funny how nothing seems to be AI’s fault.
You gdt mad, swear at it, maybe even throw it to the wall on a git of rage but, at the end of the day, deep inside you still know you screwed.
Earlier quoted context omitted.
LLMs are amazing at producing boilerplate, which removes the incentive to get rid of it. Boilerplate sucks to review. You just see a big mass of code and can't fully make sense of it when reviewing. Also, Github sucks for reviewing PRs with too many lines. So junior/mid devs are just churning boilerplate-rich code and don't really learn. The only outcome here is code quality is gonna go down very very fast.
I envy the people working at mystical places where humans were on average writing code of high quality prior LLMs. I'll never know you now.
> Software Engineering is unusual in that it works with deterministic machines. Maybe LLMs mark the point where we join our engineering peers in a world on non-determinism. Recently some people have compared LLMs to compilers and the resulting source code to object code. This is a false analogy, because compilation is (almost always) a semantics preserving transformation. LLMs are given a natural language spec (promp…
Earlier quoted context omitted.
> Every developer is using LLM Citation needed. In my circles, Senior engineer are not using them a lot, or in very specific use cases. My company is blocking LLMs use apart from a few pilots (which I am part of, and while claude code is cool, its effectiveness on a 10-year old distributed codebase is pretty low). You can't make sweeping statements like this, software engineering is a large field. And I use claude co…
Stack overflow published recently a survey in which something like 80% of developers were using AI and the rest “wants to soon”. By now I have trouble believing a competent developer is still convinced they shouldn’t use it at all , though a few ludites perhaps might hold on for a bit longer.
Earlier quoted context omitted.
It’s funny how nothing seems to be AI’s fault.
If your toaster burns your breakfast bread, Do you ultimately blame "it"? You gdt mad, swear at it, maybe even throw it to the wall on a git of rage but, at the end of the day, deep inside you still know you screwed.
Earlier quoted context omitted.
> Every developer is using LLM Citation needed. In my circles, Senior engineer are not using them a lot, or in very specific use cases. My company is blocking LLMs use apart from a few pilots (which I am part of, and while claude code is cool, its effectiveness on a 10-year old distributed codebase is pretty low). You can't make sweeping statements like this, software engineering is a large field. And I use claude co…
Stack overflow published recently a survey in which something like 80% of developers were using AI and the rest “wants to soon”. By now I have trouble believing a competent developer is still convinced they shouldn’t use it at all , though a few ludites perhaps might hold on for a bit longer.
That’s a useful tip.
Earlier quoted context omitted.
The longer term for this is "stochastic parrot". See another HN comment here comparing LLMs to theater actors or movie actors. LLMs just spew words. It just so happens that human beings can decode them into something related, useful, and meaningful surprisingly often. Might even be a useful case of pareidolia (a term I dislike, because a world without any pattern matching whatsoever would not necessarily be "better")…
I dislike the term “stochastic parrot”, because there’s plenty of evidence that LLMs do have an understanding of at least some things that they are saying. We can trace which neurons activate for a face recognition model and see that a certain neuron does light up when it sees a face. The correct features are active for the sentence “the word parrots is plural”. If you stop assuming the LLMs have no internal represen…
"Wrong answers on SAT" is also the leading hypothesis on why o3 was such an outlier - far more prone to hallucinations than either prior or following OpenAI models.
On SAT, giving a random answer is right 20% of the time - more if you ruled at least one obviously wrong answer out. Saying "I don't know" and not answering is right 0% of the time. So if you RLVR on SAT-type tests, where any answer is better than no answer, you encourage hallucinations. Hallucination avoidance in LLMs is a fragile capability, and OpenAI has probably fried its o3 with too much careless RLVR.
But another cause of hallucinations is limited self-awareness of modern LLMs. And I mean "self-awareness" in a very mechanical, no-nonsense fashion: "has information about itself and its own capabilities". LLMs have very little of that.
Humans have some awareness of the limits of their knowledge - not at all perfect, but at least there's something. LLMs get much, much less of that. LLMs learn the bulk of their knowledge from pre-training data, but pre-training doesn't teach them a lot about where the limits of their knowledge lie.
Earlier quoted context omitted.
The longer term for this is "stochastic parrot". See another HN comment here comparing LLMs to theater actors or movie actors. LLMs just spew words. It just so happens that human beings can decode them into something related, useful, and meaningful surprisingly often. Might even be a useful case of pareidolia (a term I dislike, because a world without any pattern matching whatsoever would not necessarily be "better")…
I dislike the term “stochastic parrot”, because there’s plenty of evidence that LLMs do have an understanding of at least some things that they are saying. We can trace which neurons activate for a face recognition model and see that a certain neuron does light up when it sees a face. The correct features are active for the sentence “the word parrots is plural”. If you stop assuming the LLMs have no internal represen…
It's bold to use the term "understanding" in this context. You ask it something about a topic, it gives an answer like someone who understands the topic. You change the prompt slightly, where a human who understands the topic would still give the right response trivially, the LLM outputs an answer that is both wrong/irrelevant and unpredicably and non-humanly wrong in a way that no human who exhibited understanding with the first answer could be predicted to answer the second question in the same bizarrw manner as the LLM.
The fact that the LLM can be shown to have some sort of internal representation does not necessarily mean that we should call this "understanding" in any practical sense when discussing these matters. I think it's counterproductive in getting to the heart of the matter.
> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful. This is an example of my least favorite style of feigned insight: redefining a term into meaninglessness just so you can say something that sounds different while not actu…
I’ve never liked that this behaviour is described using the term “hallucination”. If a human being talked confidently about something that they were just making up out of thin air by synthesizing based (consciously or unconsciously) on other information they know you wouldn’t call it “hallucination”: you’d call it “bullshit”. And, honestly, “bullshit” is a much more helpful way of thinking about this behaviour becaus…
The difference is that the LLM is _only that part_. In producing language as a human, I filter these words, I go back and think of new phrasings, I iterate --- in writing consciously, in speech unconsciously. So rather than a sequence it is a scattered tree filled with rhetorical dead ends, pruned through interaction with my world-model and other intellectual faculties. You can pull on one thread of words as though it were fully-formed already as a kind of Surrealist exercise (like a one-person cadavre exquis), and the result feels similar to an LLM with the temperature turned up too high.
But if nothing else, this highlights to me how easily the process of word generation may be decoupled from meaning. And it serves to explain another kind of common human experience, which feels terribly similar to the phenomenon of LLM hallucination: the "word vomit" of social anxiety. In this process it suddenly becomes less important that the words you produce are anchored to truth, and instead the language-system becomes tuned to produce any socially plausible output at all. That seems to me to be the most apt analogy.