161–170 of 233 posts
Earlier quoted context omitted.
That's not just small, it's utterly miniscule. It's most certainly not large.
Depends. 3k is pretty much enough for a fully-featured XY. So no context, and differences of the definition of "large". Perhaps if you come from Java, then yeah. shrugs
Earlier quoted context omitted.
I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. Sure it makes mistakes occasionally but usually it's a no-brainer to fix them.
> I use LLMs for writing generic, repetitive code, like scaffolding. It's OK with boring, generic stuff. In other words, they're OK in use-cases that programmers need to eliminate , because it means there's high demand for a reusable library, some new syntax sugar, or an improved API.
Earlier quoted context omitted.
Testing is better than nothing, but still highly fallible. Take these winning examples from the underhanded C contest [0], [1], where the issues are completely innocuous mistakes that seem to work perfectly despite completely undermining the nominal purpose of the code. You can't substitute an automated process for thinking deeply and carefully about the code. [0] https://www.underhanded-c.org/#winner [1] https://www…
I think it is unlikely (of course not impossible) an LLM would fail in that way. The underhanded C contest is not a case of people accidentally producing highly misleading code, it is a case of very smart people going to a great amount of effort to intentionally do that. Most of the time, if your code is wrong, it doesn't work in some obvious way – it doesn't compile, it fails some obvious unit tests, etc. Code accid…
Let's assume that 1 in 10,000 coding sessions produce an innocuous, test-passing function that's catastrophically wrong. If you have a mid to large size company with 1000 devs doing two sessions a day, you'll see one of these a week within that single company. Actually sounds a lot like the IoT industry now that I've written it.
Earlier quoted context omitted.
A language like Golang tries really hard to only have _one_ way to do something, one right way, one way. Just one way. See how it was before generics. You just have a for loop. Can't really mess up a for loop. I predict that the variance in success in using LLM for coding (even agentic coding with multi-step rather than a simple line autosuggest or block autosuggest that many are familar with via CoPilot) has much mo…
> A language like Golang tries really hard to only have _one_ way to do something Really? Logging in Go: A Comparison of the Top 9 Libraries https://betterstack.com/community/guides/logging/best-golang...
I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.
GPT can't even tell what its done or give what it knows it should. It's endless, "Apologies, here is what you actually asked for ..." and again it isn't.
I've got a lot of hallucinations like that from LLMs. I really don't get how so many people can get LLMs to code most of their tasks without those issues permanently popping up.
GPT can't even tell what its done or give what it knows it should. It's endless, "Apologies, here is what you actually asked for ..." and again it isn't.
Earlier quoted context omitted.
I agree with you but as I've matured as a programmer, I feel like it's very hard to get abstractions for boilerplate right. Every library I've seen attempt to do it has struggled.
Even Rails went with codegen despite being backed by the language most amenable to abstractions.
Earlier quoted context omitted.
What I've found is that the quality of an AI answer is inversely proportional to the knowledge of the person reading it. To an amateur it answers expertly, to an expert it answers amateurishly. So no, it's not a lack of skill in prompting: I've sat down with "prompting" "experts" and universally they overlook glaring issues when assessing the how good an answer it was. When I tell them where to press it further it br…
I know what I want to do and how to do it (expert), so the results are good, for me at least. Of course I have to polish it off here and there.
Earlier quoted context omitted.
I think it is unlikely (of course not impossible) an LLM would fail in that way. The underhanded C contest is not a case of people accidentally producing highly misleading code, it is a case of very smart people going to a great amount of effort to intentionally do that. Most of the time, if your code is wrong, it doesn't work in some obvious way – it doesn't compile, it fails some obvious unit tests, etc. Code accid…
The examples are just a sort of existence proof rather than a comment on exactly how LLM code can fail. I think this is something to consider when you put the scale of usage into perspective. Let's assume that 1 in 10,000 coding sessions produce an innocuous, test-passing function that's catastrophically wrong. If you have a mid to large size company with 1000 devs doing two sessions a day, you'll see one of these a…
Earlier quoted context omitted.
I’m going to double down on this one: an LLM is only as good as its training data. A hallucination to me is an invented piece of information, here it’s going on something real that it’s seen. To me that’s at best contamination, at worst an adversarial attack - something that’s been planted in the data. Here this is obviously not the case, which is why I said “more in common with” instead of “is” above. Semantics perh…
It’s been trained to produce valid code, fed millions of examples, and in this case it’s outputting invented syntax. Whether there’s an example in its training data, it’s still a hallucination and shouldn’t have been output since it’s not valid.