Live data from Hacker News

Generative AI coding tools and agents do not work for me

blog.miguelgrinberg.com

391–400 of 464 posts

Re: Generative AI coding tools and agents do not work for me

#391
post #332

Earlier quoted context omitted.

So, you're saying you provided examples of the code and APIs and more, in the context window, and it succeeds? That sounds very much unlike the post I responded to, which claimed "no knowledge". You're also seemingly missing this: "If you're getting answers, it has seen it elsewhere" The context window is 'elsewhere'.

If that's the distinction you're drawing then it's totally meaningless in the context of the question of where the information is going to come from if not Stack Overflow. We're never in a situation where we're using an open source library that has zero information about it: The code is by definition available to be put in the context window. As they say, it sounds like you're technically correct, which is the best k…

The argument is futile as the goal posts move constantly. In one moment the assertion is it’s just megacopy paste, then the next when evidence is shown that it’s able to one shot construct seemingly novel and correct answers from an api spec or grammar never seen before, the goal posts move to “it’s unable to produce results on things it’s never been trained on or in its context” - as if making up a fake language and asking it write code in it and its inability to do so without a grammar is an indication of literally anything.

To anyone who has used these tools in anger it’s remarkable given they’re only trained on large corpuses of language and feedback they’re able to produce what they do. I don’t claim they exist outside their weights, that’s absurd. But the entire point of non linear function activations with many layers and parameters is to learn highly complex non linear relationships. The fact they can be trained as much as they are with as much data as they have without overfitting or gradient explosions means the very nature of language contains immense information in its encoding and structure, and the network by definition of how it works and is trained does -not- just return what it was trained on. It’s able to curve fit complex functions that inter relate semantic concepts that are clearly not understood as we understand them, but in some ways it represents an “understanding” that’s sometimes perhaps more complex and nuanced than even we can.

Anyway the stochastic parrot euphemism misses the point that parrots are incredibly intelligent animals - which is apt since those who use that phrase are missing the point.

Re: Generative AI coding tools and agents do not work for me

#392

Earlier quoted context omitted.

ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. What do you think will happen when everyone is using the AI tools to answer their questions? We'll be back in the world of Encyclopedias, in which central authorities spent large amounts of money manually collecting information and publishing it. And then they spent a good amount o…

> ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. I mean... They also can read actual documentation. If I'm working on any api work or a language I'm not familiar with, I ask the LLM to include the source they got their answer from and use official documentation when possible. That lowers the hallucination rate significantly and…

> I mean... They also can read actual documentation.

This 100%. I use o3 as my primary search engine now. It is brilliant at finding relevant sources, summarising what is relevant from them, and then also providing the links to those sources so I can go read them myself. The release of o3 was a turning point for me where it felt like these models could finally go and fetch information for themselves. 4o with web search always felt inadequate, but o3 does a very good job.

> In theory, all stackoverflow answers are just regurgitated documentation, no?

This is unfair to StackOverflow. There is a lot of debugging and problem solving that has happened on that platform of undocumented bugs or behaviour.

Re: Generative AI coding tools and agents do not work for me

#393
post #238

Earlier quoted context omitted.

It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. That doesn't mean it knows the answer . That means it guessed or hallucinated correctly. Guessing isn't knowing. edit: people seem to be missing my point, so let me rephrase. Of course AIs don't think, but that wasn't what I was…

>It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. How would you reconcile this with the fact that SOTA models are only a few TB in size? Trained on exabytes of data, yet only a few TB in the end. Correct answers couldn't be dumb luck either, because otherwise the models would…

> How would you reconcile this with the fact that SOTA models are only a few TB in size? Trained on exabytes of data, yet only a few TB in the end.

This is false. You are off by ~4 orders of magnitude by claiming these models are trained on exabytes of data. It is closer to 500TB of more curated data at most. Contrary to popular belief LLMs are not trained on "all of the data on the internet". I responded to another one of your posts that makes this false claim here:

https://news.ycombinator.com/item?id=44283713

Re: Generative AI coding tools and agents do not work for me

#394
post #376

Earlier quoted context omitted.

Human knowledge != Reddit/Twitter/Wikipedia

Conversely, what do you posit is part of human knowledge but isn't scrapable from the internet?

I mean, as far as a corpus goes, I suppose all text on the internet gets pretty close if most books are included, but even then you’re mostly looking at English language books that have been OCR’d.

But I look down my nose at conceptions that human knowledge is packagable as plain text, our lives, experience, and intelligence is so much more than the cognitive strings we assemble in our heads in order to reason. It’s like in that movie Contact when Jodie Foster muses that they should have sent a poet. Our empathy and curiosity and desires are not encoded in UTF8. You might say these are realms other than knowledge, but woe to the engineer who thinks they’re building anything superhuman while leaving these dimensions out, they’re left with a cold super-rationalist with no impulse to create of its own.

Re: Generative AI coding tools and agents do not work for me

#395

> Another common argument I've heard is that Generative AI is helpful when you need to write code in a language or technology you are not familiar with. To me this also makes little sense. I'm not sure I get this one. When I'm learning new tech I almost always have questions. I used to google them. If I couldn't find an answer I might try posting on stack overflow. Sometimes as I'm typing the question their search wo…

ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. What do you think will happen when everyone is using the AI tools to answer their questions? We'll be back in the world of Encyclopedias, in which central authorities spent large amounts of money manually collecting information and publishing it. And then they spent a good amount o…

We'll start writing documentation for primary consumption by LLMs rather than human readers. The need for sites like SO will not vanish overnight but it will diminish drastically.

Re: Generative AI coding tools and agents do not work for me

#396
post #238

Earlier quoted context omitted.

It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. That doesn't mean it knows the answer . That means it guessed or hallucinated correctly. Guessing isn't knowing. edit: people seem to be missing my point, so let me rephrase. Of course AIs don't think, but that wasn't what I was…

It's just Pattern matching. Most APIs, and hell, most code is not unique or special. Its all been done a thousands of times before. Thats why an LLM can be helpful on some tool you've written just for yourself and never released anywhere. As to 'knows the answer', I'm don't even know what that means with these tools. All I know is if it is helpful or not.

'Pattern matching' isn't just all you need, it's all there is.

Re: Generative AI coding tools and agents do not work for me

#397

> Another common argument I've heard is that Generative AI is helpful when you need to write code in a language or technology you are not familiar with. To me this also makes little sense. I'm not sure I get this one. When I'm learning new tech I almost always have questions. I used to google them. If I couldn't find an answer I might try posting on stack overflow. Sometimes as I'm typing the question their search wo…

I sort of disagree with this argument in TFA, as you say, though the rest of the article highlights a limitation. If I'm unfamiliar with the API, I can't judge whether the answer is good.

There is a sweet spot of situations I know well enough to judge a solution quickly, but not well enough to write code quickly, but that's a rather narrow case.

Re: Generative AI coding tools and agents do not work for me

#398

Earlier quoted context omitted.

I only generate the code once with GenAI and typically fix a bug or two - or at worst use its structure. Rarely do I toss a full PR. It’s interesting some folks can use them to build functioning systems and others can’t get a PR out of them.

The problem is that at this stage we mostly just have people's estimates of their own success to go on, and nobody thinks they're incompetent. Nobody's going to say "AI works really well for, me but I just pump out dross my colleagues have to fix" or "AI doesn't work for me but I'm an unproductive, burnt out hack pretending I'm some sort of craftsman as the world leaves me behind". This will only be resolved out ther…

I aspire to live up to your description of the best sort of developer. But I think there might also be a danger that that approach can turn into an excuse for spending the week overthinking (possibly while goofing off as well; I've done it), then writing a first cut on Friday, leaving no time for the multiple iterations that are often necessary to get to the best solution. In other words, I think sometimes it's necessary to just start coding sooner than we'd like so we can start iterating toward the right solution. But that "unproductive, burnt out hack" line hits a bit too close to home for me these days, and I'm starting to entertain the possibility that an LLM-based agent might have more energy for doing those multiple iterations than I do.

Re: Generative AI coding tools and agents do not work for me

#399

> Another common argument I've heard is that Generative AI is helpful when you need to write code in a language or technology you are not familiar with. To me this also makes little sense. I'm not sure I get this one. When I'm learning new tech I almost always have questions. I used to google them. If I couldn't find an answer I might try posting on stack overflow. Sometimes as I'm typing the question their search wo…

ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. What do you think will happen when everyone is using the AI tools to answer their questions? We'll be back in the world of Encyclopedias, in which central authorities spent large amounts of money manually collecting information and publishing it. And then they spent a good amount o…

> ChatGPT and Gemini literally only know the answer because they read StackOverflow

Obviously this isn’t true. You can easily verify this by inventing and documenting an API and feeding that description to an LLM and asking it how to use it. This works well. LLMs are quite good at reading technical documentation and synthesizing contextual answers from it.

Re: Generative AI coding tools and agents do not work for me

#400
post #332

Earlier quoted context omitted.

This is wrong. I write toy languages and frameworks for fun. These are APIs that simply don't exist outside of my code base, and LLMs are consistently able to: * Read the signatures of the functions. * Use the code correctly. * Answer questions about the behavior of the underlying API by consulting the code. Of course they're just guessing if they go beyond what's in their context window, but don't underestimate cont…

So, you're saying you provided examples of the code and APIs and more, in the context window, and it succeeds? That sounds very much unlike the post I responded to, which claimed "no knowledge". You're also seemingly missing this: "If you're getting answers, it has seen it elsewhere" The context window is 'elsewhere'.

This is moving goalposts vs the original claim upthread that LLMs are just regurgitating human-authored stackoverflow answers and without those answers it would be useless.

It’s silly to say that something LLMs can reliably do is impossible and every time it happens it’s “dumb luck”.

Post reply on HN