Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't fall into the anti-AI hype

antirez.com

Post reply on HN
Print view View original

941–950 of 1001 posts

Re: Don't fall into the anti-AI hype

#941

Post by JohnMakin »

dividedbyzero wrote: post #823
unyttigfjelltol wrote: post #604

Earlier quoted context omitted.

> non-trivial coding tasks I’ve come back to the idea LLMs are super search engines. If you ask it a narrow, specific question, with one answer, you may well get the answer. For the “non-trivial” questions, there always will be multiple answers, and you’ll get from the LLM all of these depending on the precise words you use to prompt it. You won’t get the best answer, and in a complex scenario requiring highly recurs…

They don't even really do that IME. If I ask Claude or ChatGPT to generate terraform for non-trivial but by no means obscure or highly unusual setups, they almost invariably hallucinate part of the answer even if a documented solution exists that isn't even that difficult. Maybe vibe coding JavaScript is that much better, or I'm just hopeless at prompting, but I feel a few dozen lines of fairly straightforward terraf…

For better or for worse have spent a large amount of time in terraform since 0.13 and I can confidently say LLM's are very, very bad at it. My favorite is when it invents internal functions (that look suspiciously like python) that do not exist, even when corrected, it will still keep going back to them. A year or two ago there were bad problems with hallucinated resource field names but I haven't seen that as much these days.

It however, is pretty good at refactoring given a set of constraints and an existing code base. It is decent at spitting out boilerplate code for well-known resources (such as AWS), but then again, those boilerplate examples are mostly coming straight from the documentation. The nice thing about refactoring with LLM's in terraform is, even if you vibe it, the refactor is trivially verifiable because the plan should show no changes, or the exact changes you would expect.

Re: Don't fall into the anti-AI hype

#942

Post by hansvm »

totallykvothe wrote: post #569

I don't understand the stance that AI currently is able to automate away non-trivial coding tasks. I've tried this consistently since GPT 3.5 came out, with every single SOTA model up to GPT 5.1 Codex Max and Opus 4.5. Every single time, I get something that works, yes, but then when I start self-reviewing the code, preparing to submit it to coworkers, I end up rewriting about 70% of the thing. So many important deta…

You alluded to it, but also:

3) Not everyone codes the same things

4) It's easy to get too excited about the tech and ignore its failure modes when describing your experiences later

I use AI a lot. With your own control plane (as opposed to a generic Claude Code or whatever) you can fully automate a lot more things. It's still fundamentally incapable of doing tons of tasks though at any acceptable quality level, and I strongly suspect all of (2,3,4) are guiding the disconnect you're seeing.

Take the two things I've been working on this morning as an example.

One was a one-off query. I told it the databases it should consider, a few relevant files, roughly how that part of the business works, and asked it to come back when it finished. When it was done I had it patch up the output format. It two-shot (with a lot of helpful context) something that would have taken me an hour or more.

Another is more R&D-heavy. It pointed me to a new subroutine I needed (it couldn't implement it correctly though) and is otherwise largely useless. It's actively harmful to have it try to do any of the work.

It's possible that (1) matters more than you suspect too. AI has certain coding patterns it likes to use a lot which won't work in my codebase. Moreover, it can't one-shot the things I want. It can, however, follow a generic step-by-step guide for generating those better ideas, translating worse ideas into things that will be close enough to what I need, identifying where it messed up, and refactoring into something suitable, especially if you take care to keep context usage low and whatnot. A lot of people seem to be able to get away with CLAUDE.md or whatever, but I like having more granular control of what the thing is going to be doing.

Re: Don't fall into the anti-AI hype

#943

Post by locknitpicker »

totallykvothe wrote: post #569

I don't understand the stance that AI currently is able to automate away non-trivial coding tasks. I've tried this consistently since GPT 3.5 came out, with every single SOTA model up to GPT 5.1 Codex Max and Opus 4.5. Every single time, I get something that works, yes, but then when I start self-reviewing the code, preparing to submit it to coworkers, I end up rewriting about 70% of the thing. So many important deta…

> Every single time, I get something that works, yes, but then when I start self-reviewing the code, preparing to submit it to coworkers, I end up rewriting about 70% of the thing.

You might want to review how you approach these tools. Complaining that you need to rewrite 70% of the code screams of poor prompting, with too vague inputs, no constraints, and no feedback at all.

Using agents to help you write code is far from a one-shot task, but if throwing out 70% of what you create screams out that you are prompting the agent to create crap.

> 1) I'm not good at prompting, even though I am one of the earliest AI in coding adopters I know, and have been consistent for years. So I find this hard to accept.

I think you need to take a humble pill, review how you are putting together these prompts, figure out what you are doing wrong in prompts and processes, and work up from where you are at this point. If 70% of your output is crap, the problem is in your input.

I recommend you spend 20 minutes with your agent of choice prompting it to help you improve your prompts. Check instruction files, spec-driven approaches, context files, etc. Even a plain old README.md helps a lot. Prompt your agent to generate it for you. From there, instead of one-shot prompts try to break down a task into multiple sub steps with small deliverables. Always iterate on your instruction files. It you spend a few minutes on this, you will quickly halve your churn rate.

Re: Don't fall into the anti-AI hype

#944

Post by jplusequalt »

theshrike79 wrote: post #819
zeroonetwothree wrote: post #311

Earlier quoted context omitted.

You’re right of course. For me there’s no flow state possible with LLM “coding”. That makes it feel miserable instead of joyous. Sitting around waiting while it spits out tokens that I then have to carefully look over and tweak feels like very hard work. Compared to entering flow and churning out those tokens myself, which feels effortless once I get going. Probably other people feel differently.

Why do you feel you need to "carefully look over and tweak" stuff? Can you define code quality and the goal of the program in a deterministic way? If it quacks like a duck, walks like a duck and is a duck, does it matter if it's actually a raven inside?

Yes, if your goal is to build a duck, and to understand what goes into building a duck. A lot of people derive joy from learning how to do something, not merely seeing the end result.

Re: Don't fall into the anti-AI hype

#945

Post by parliament32 »

simonw wrote: post #939
parliament32 wrote: post #918

Earlier quoted context omitted.

Not even coding tasks. Just getting an LLM to help me put together a PromQL query to do something somewhat non-standard takes dozens of tries and copy/pasting back error messages.. and these aren't complex errors, trivial things like missing closing brackets and the like. I know the usual clap back is "you're just missing this magical workflow" or "you need to prompt better" but.. do I really need to prompt "make sur…

Yes, you're missing a magic workflow. If you find yourself having to copy and paste errors back and forward you need to upgrade to a coding agent harness like Claude Code so the LLM can try things out and then fix the errors on its own. If you're not willing to do that you can also fix this by preparing a text file with a few examples of correctly formatted queries and pasting that in at the start of your session, or…

So, let me get this straight, LLMs need a "coding agent harness" to figure out that they need to close brackets? Wild.

Re: Don't fall into the anti-AI hype

#946

Post by dist-epoch »

smcl wrote: post #855
dist-epoch wrote: post #83

Earlier quoted context omitted.

People said the exact same thing about (numbers from memory, might be off): - when Google paid $1 bil for YouTube - when Facebook paid $1 bil for Instagram - when Facebook paid $1 bil for WhatsApp The same thing - these 3 companies make no money, and have no path to making money, and that the price paid was crazy and decoupled from any economics. Yet now, in hindsight, they look like brilliant business decisions.

While many people thought Facebook/Google paid too much for these companies, you're making an apples-to-oranges comparison. That part about there being "no path to making money" is wrong - online advertising was a huge industry and only getting stronger and while YT/Insta/Whatsapp may have struggled as standalone companies it was clear they'd unlock an enormous amount of value as part of a bigger company that already…

> Not to mention the logical fallacy at the core of your point

Yes, it's a logical fallacy. Another one is saying "I don't see any viable business model, therefore there is no viable business model".

Blast from the past:

> YouTube is a content paradise though. There's tons of value there and you can sell ads against it or even charge for premium services.

> Where's the money in Instagram? The content is practically worthless and their only real value is in their userbase. Even though I use the Instagram client, most of the time I see photos, they come through Twitter. So that also reinforces for me that any value is in the users and not the actual content, which is mostly crap.

> I'm more convinced that we're in a 2nd bubble now more than ever.

https://news.ycombinator.com/item?id=3818037

Another one:

> Does anyone else think this valuation is insane? It's like $300/registered user. The company doesn't have a business model. No way the handful of employees are worth $1B. My mind is blown.

https://news.ycombinator.com/item?id=3817930

Re: Don't fall into the anti-AI hype

#947

Post by dist-epoch »

Rooster61 wrote: post #904

The question that I haven't seen answered yet is whether or not we will reach a sort of "peak vibe coding" phase. What I mean by that is, right now, LLM's are somewhat decent at writing workable code. That code, however, needs babysitting to keep from going off the rails. And that code is sourced from training, which has been gleaned from the billions of lines of code written by hackers everywhere and pushed to sourc…

The first generation of AlphaGo was trained on human-human games.

The second genration removed that, was trained entirely on computer generated games.

Exactly because human data is running out, synthetic data is very big right now in all AI labs.

Re: Don't fall into the anti-AI hype

#950

Post by OkayPhysicist »

concats wrote: post #778
daxfohl wrote: post #599

Earlier quoted context omitted.

Agreed, but: There's been a notable jump over the course of the last few months, to where I'd say it's inevitable. For a while I was holding out for them to hit a ceiling where we'd look back and laugh at the idea they'd ever replace human coders. Now, it seems much more like a matter of time. Ultimately I think over the next two years or so, Anthropic and OpenAI will evolve their product from "coding assistant" to "…

> Ultimately I think over the next two years or so, Anthropic and OpenAI will evolve their product from "coding assistant" to "engineering team replacement" The way I see it, there will always be a layer in the corporate organization where someone has to interact with the machine. The transitioning layer from humans to AIs. This is true no matter how high up the hierarchy you replace the humans, be it the engineers l…

My suspicion is that it will be bad for salaries, mostly because it'll kill the "looks difficult" moat that software development currently has. Developers know that "understanding source code" is far from the hard part of developing software, but non-technical folks' immediate recoiling in the face of the moon runes has kept our profession pretty easy to justify high pay for for ages. If our jobs transition to largely "communing with the machines", then we'll go from a "looks hard, is hard" job, to a "looks easy, is hard" job, which historically hurts bargaining power.
Post reply on HN