Live data from Hacker News

Generative AI coding tools and agents do not work for me

blog.miguelgrinberg.com

361–370 of 464 posts

Re: Generative AI coding tools and agents do not work for me

#361

So far as I can tell, generative AI coding tools make the easy part of the job go faster, without helping with the hard part of the job - in fact, possibly making it harder. Coding just doesn't take that much time, and I don't need help doing it. You could make my coding output 100x faster without materially changing my overall productivity, so I simply don't bother to optimize there.

No software engineer needs any help if they keep working in the same stack and problem domain that they already know front to back after a few years doing the same thing. They wouldn't need any coding tool even. But that a pretty useless thing to say. To each their own.

Re: Generative AI coding tools and agents do not work for me

#362
That is my experience with Windsurf / Cursor type tools. Faster for some things but generally super annoying and slow.

The Codex workflow however really is a game changer imo. It takes the time to ensure changes are consistent with other code and the async workflow is just so much nicer.

Re: Generative AI coding tools and agents do not work for me

#364
The author makes the excellent point that LLM-coding still has a human bottleneck at the code review point - regardless of whether the issue at hand is fixed or not.

Leaving aside the fact that this isn't an LLM problem; we've always had tech debt due to cowboy devs and weak management or "commercial imperatives":

I'd be interested to know if any of the existing LLM ELO style leaderboards mark for code quality in addition to issue fixing?

The former seems a particularly useful benchmark as they become more powerful in surface abilities.

Re: Generative AI coding tools and agents do not work for me

#365
post #238

Earlier quoted context omitted.

> ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. I know this isn't true because I work on an API that has no answers on stackoverflow (too new), nor does it have answers anywhere else. Yet, the AI seems to able to accurately answer many questions about it. To be honest I've been somewhat shocked at this.

It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck. That doesn't mean it knows the answer . That means it guessed or hallucinated correctly. Guessing isn't knowing. edit: people seem to be missing my point, so let me rephrase. Of course AIs don't think, but that wasn't what I was…

>It is absolutely true, and AI cannot think, reason, comprehend anything it has not seen before. If you're getting answers, it has seen it elsewhere, or it is literally dumb, statistical luck.

How would you reconcile this with the fact that SOTA models are only a few TB in size? Trained on exabytes of data, yet only a few TB in the end.

Correct answers couldn't be dumb luck either, because otherwise the models would pretty much only hallucinate (the space of wrong answers is many orders of magnitude larger than the space of correct answers), similar to the early proto GPT models.

Re: Generative AI coding tools and agents do not work for me

#366
post #276

Earlier quoted context omitted.

> when Google used to do the same thing for you way before "AI"? Which is never? Do you often just lie to win arguments? LLM gives you a synthesized answer, search engine only returns what already exists. By definition it can not give you anything that is not a super obvious match

> Which is never? In my experience it was "a lot". Because my stack traces were mostly hardware related problems on arm linux in that period. But I suppose your stack traces were much different and superior and no one can have stack traces that are different from yours. The world is composed of just you and your project. > Do you often just lie to win arguments? I do not enjoy being accused of lying by someone stuck…

According to a quick search on google, which is not very useful these days, the maximum query length is 32 words or 2000 characters and change depending on which answer you trust.

Whatever it is specifically, the idea that you could just paste a 600 line stack trace unmodified into google, especially "way before AI" and get pointed to the relevant bit for your exact problem is obviously untrue.

Re: Generative AI coding tools and agents do not work for me

#367
One thing about AI that I feel like no company that is already inserting into their workforce is thinking about is what the future looks like when your company depends on it. If AI is doing the work that junior employees used to do, then you are losing the base knowledge that your employees used to learn. Maybe in the coming years it starts to take over more and more roles that people used to do and companies can decrease their workforce. AI comes a lot cheaper than real people (at least that's the selling point).

Most tech companies however tend to operate following a standard enshittification schedule. First they are very cheap, supported by investments and venture capitalists. Then they build a large user base who become completely dependent on them as alternatives disappear (in this case as they lose the institutional knowledge that their employees used to have). Then they seek to make money so the investors can make their profits. In this case I could see the cost of AI rising a lot, after companies have already built it in to their business. AI eventually has to start making money. Just like Amazon had to, and Facebook, and Uber, and Twitter, and Netflix, etc.

From all the talk I see of companies embracing AI wholeheartedly it seems like they aren't looking any further than the next quarter. It only costs so much per month to replace so many man hours of work! I'm sure that won't last once AI is deeply embedded into so many businesses that they can start charging whatever they want to.

Re: Generative AI coding tools and agents do not work for me

#368

> Another common argument I've heard is that Generative AI is helpful when you need to write code in a language or technology you are not familiar with. To me this also makes little sense. I'm not sure I get this one. When I'm learning new tech I almost always have questions. I used to google them. If I couldn't find an answer I might try posting on stack overflow. Sometimes as I'm typing the question their search wo…

ChatGPT and Gemini literally only know the answer because they read StackOverflow. Stack Overflow only exists because they have visitors. What do you think will happen when everyone is using the AI tools to answer their questions? We'll be back in the world of Encyclopedias, in which central authorities spent large amounts of money manually collecting information and publishing it. And then they spent a good amount o…

I broadly agree that cutting new knowledge will need to continue being done and that overuse of LLMs could undermine that, yet... When was the last time you paid to read an APIs' docs? It costs money for companies to make those too.

Re: Generative AI coding tools and agents do not work for me

#369

Earlier quoted context omitted.

Of course that's what the industry is selling because they want to make money. Yes, it's easy to create a proof of concept but once you get out of greenfield into 50-100k tokens needed in the context (reading multiple 500 line files, thinking, etc) the quality drops and you need to know how to focus the models to maintain the quality. "Write me a server in Go" only gets you so far. What is the auth strategy, what end…

> What is the auth strategy, what endpoints do you need, do you need to integrate with a library or API, are there any security issues, how easy is the code to extend, how do you get it to follow existing patterns? That is software engineering realm, not using LLMs realm. You have to answer all of these questions even with traditional coding. Because they’re not coding questions, they’re software design questions. An…

Agreed, but people sell "vibe coding" without acknowledging you need more than vibes.

LLMs can help answer the questions. However, they're not going to necessarily make the correct choices or implementation without significant input from the user.

Re: Generative AI coding tools and agents do not work for me

#370
post #208

Earlier quoted context omitted.

Is it a skill worth learning though? How much does the output quality improve? How transferable is it across models and tools of today, and of the future? From what I see of AI programming tools today, I highly doubt the skills developed are going to transfer to tools we'll see even a year from now.

Given I see people insisting these tools don't work for them at all, and some of my results recently include spitting out a 1k line API client with about 5 brief paragraphs of prompts, and designing a website (the lot, including CSS, HTML, copy, database access) and populating the directory on it with entries, I'd think the output quality improves a very great deal. From what I see of the tools, I think the skills de…

If I had a green field project that was low novelty I would happily use AI to get a prototype out the door quickly. I basically never work on those kinds of projects though, and I've seen AI tools royal screw up enough times given clear direction on both novel and trivial tasks in existing code bases.

Detailed specs are certainly a transferable skill, what isn't is the tedious hand holding and defensive prompting. In my entire career I've worked with a lot of people, only one required as much hand holding as AI. That person was using AI to do all their work.

Post reply on HN