Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Generative AI coding tools and agents do not work for me

blog.miguelgrinberg.com

Post reply on HN
Print view View original

331–340 of 464 posts

Re: Generative AI coding tools and agents do not work for me

#331

Post by jamesrcole »

rainonmoon wrote: post #325
jumploops wrote: post #273

Earlier quoted context omitted.

> It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. The amazing thing about LLMs is that we still don’t know how (or why) they work! Yes, they’re magic mirrors that regurgitate the corpus of human knowledge. But as it turns out, most human knowledge is already regurgitation (see: the patent system). Novelty is rare, and LLMs have an incredible ability to pattern match and…

> the corpus of human knowledge. Goodness this is a dim view on the breadth of human knowledge.

what do you object to about it? I don't see an issue with referring to "the corpus of human knowledge". "Corpus" pretty much just means the "collection of".

Re: Generative AI coding tools and agents do not work for me

#332

Post by b112 »

demosthanos wrote: post #330
b112 wrote: post #238

Earlier quoted context omitted.

It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. That doesn't mean it knows the answer . That means it guessed or hallucinated correctly. Guessing isn't knowing. edit: people seem to be missing my point, so let me rephrase. Of course AIs don't think, but that wasn't what I was…

This is wrong. I write toy languages and frameworks for fun. These are APIs that simply don't exist outside of my code base, and LLMs are consistently able to: * Read the signatures of the functions. * Use the code correctly. * Answer questions about the behavior of the underlying API by consulting the code. Of course they're just guessing if they go beyond what's in their context window, but don't underestimate cont…

So, you're saying you provided examples of the code and APIs and more, in the context window, and it succeeds? That sounds very much unlike the post I responded to, which claimed "no knowledge". You're also seemingly missing this:

"If you're getting answers, it has seen it elsewhere"

The context window is 'elsewhere'.

Re: Generative AI coding tools and agents do not work for me

#333

Post by elbear »

nottorp wrote: post #243
addandsubtract wrote: post #232

Earlier quoted context omitted.

Not if you prompt it to explain the answer it gives you.

Not the same thing. Copying code, even with comprehensive explanations, teaches less than writing/adjusting your own code based on advice.

It can also do that if you ask it. It can give you exercises that you can solve. But you have to specifically ask, because by default it just gives you code.

Re: Generative AI coding tools and agents do not work for me

#334

Post by giantg2 »

The true test is can it write tests? Ask the dev if they use it to write tests. The answers to #1 is it can't really. The answer to #2 should be no.

AI can write some tests, but it can't design thorough ones. Perhaps the best way to use AI is to have a human writing thorough and well documented tests as part of TDD, asking AI to write code to meet those tests, then thoroughly reviewing that code.

AI saves me just a little time by writing boilerplate stuff for me, just one step above how IDEs have been providing generated getters and setters.

Re: Generative AI coding tools and agents do not work for me

#335

Post by npilk »

notnullorvoid wrote: post #120
waprin wrote: post #19

To some degree, traditional coding and AI coding are not the same thing, so it's not surprising that some people are better at one than the other. The author is basically saying that he's much better at coding than AI coding. But it's important to realize that AI coding is itself a skill that you can develop. It's not just , pick the best tool and let it go. Managing prompts and managing context has a much higher ski…

Is it a skill worth learning though? How much does the output quality improve? How transferable is it across models and tools of today, and of the future? From what I see of AI programming tools today, I highly doubt the skills developed are going to transfer to tools we'll see even a year from now.

Maybe this is yet another application of the bitter lesson. It's not worth learning complex processes for partnering with AI models, because any productivity gains will pale in comparison to the performance improvement from future generations.

Re: Generative AI coding tools and agents do not work for me

#336

Post by nottorp »

elbear wrote: post #333
nottorp wrote: post #243

Earlier quoted context omitted.

Not the same thing. Copying code, even with comprehensive explanations, teaches less than writing/adjusting your own code based on advice.

It can also do that if you ask it. It can give you exercises that you can solve. But you have to specifically ask, because by default it just gives you code.

Of course, I originally was picking on Stack Overflow's moderation.

Which strongly discouraged trying to teach people.

Re: Generative AI coding tools and agents do not work for me

#337

Post by cardanome »

skydhash wrote: post #311
anelson wrote: post #299

Earlier quoted context omitted.

I’ve experienced this as well. If management is not competent they can’t tell (or don’t want to hear) when a “star” performer is actually a very expensive wrapper around a $20/mo cursor subscription. Unlike the author of the article I do get a ton of value from coding agents, but as with all tools they are less than useless when wielded incompetently. This becomes more damaging in an org that already has perverse inc…

Git blame can do a lot in those situations. Find the general location of the bug, then assign everyone that has touched it to the ticket.

Is that really something you are doing in your job?

Most of my teams have been very allergic to assigning personal blame and management very focused on making sure everyone can do everything and we are always replaceable. So maybe I could phrase it like "X could help me with this" but saying X is responsible for the bug would be a no no.

Re: Generative AI coding tools and agents do not work for me

#338

Post by swader999 »

jasode wrote: post #235
nottorp wrote: post #222

Earlier quoted context omitted.

You remember when Google used to do the same thing for you way before "AI"? Okay, maybe sometimes the post about the stack trace was in Chinese, but a plain search used to be capable of giving the same answer as a LLM. It's not that LLMs are better, it's search that got entshittified.

>You remember when Google used to do the same thing for you way before "AI"? [...] stack trace [...], but a plain search used to be capable of giving the same answer as a LLM. The "plain" Google Search before LLM never had the capability to copy&paste an entire lengthy stack trace (e.g. ~60 frames of verbose text) because long strings like that exceeds Google's UI. Various answers say limit of 32 words and 5784 chara…

Those truly were the dark ages. I don't know how people did it. They were a different breed.

Re: Generative AI coding tools and agents do not work for me

#339

Post by demosthanos »

b112 wrote: post #332
demosthanos wrote: post #330

Earlier quoted context omitted.

This is wrong. I write toy languages and frameworks for fun. These are APIs that simply don't exist outside of my code base, and LLMs are consistently able to: * Read the signatures of the functions. * Use the code correctly. * Answer questions about the behavior of the underlying API by consulting the code. Of course they're just guessing if they go beyond what's in their context window, but don't underestimate cont…

So, you're saying you provided examples of the code and APIs and more, in the context window, and it succeeds? That sounds very much unlike the post I responded to, which claimed "no knowledge". You're also seemingly missing this: "If you're getting answers, it has seen it elsewhere" The context window is 'elsewhere'.

If that's the distinction you're drawing then it's totally meaningless in the context of the question of where the information is going to come from if not Stack Overflow. We're never in a situation where we're using an open source library that has zero information about it: The code is by definition available to be put in the context window.

As they say, it sounds like you're technically correct, which is the best kind of correct. You're correct within the extremely artificial parameters that you created for yourself, but not in any real world context that matters when it comes to real people using these tools.

Post reply on HN