Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Honest Review of AI Programming

mropert.github.io

Post reply on HN
Print view View original

51–60 of 90 posts

Re: An Honest Review of AI Programming

#51

Post by saghm »

Sharlin wrote: post #8
daishi55 wrote: post #3

> “Agentic workflow” (or whatever they’re calling it at the time you’re reading this article) > Unlike the silver bullets of the past (like microservices or NoSQL) > Hallucinations are an inherent property of how LLMs work. > It’s all marketing and buzzwords Not a serious article or thinker. I can get this stuff on Reddit if I want to read thrice-regurgitated cliches about AI.

I'd like to hear actual rebuttals to those quotes.

Programming by hand also empircally leads to quite a lot of bugs. The actual calculus for whether these tools are worth it to solve a given problem is as function of how much time it takes to reach the desired level of quality compared to writing by hand. I struggle to see why "they hallucinate and aren't deterministic" is enough to conclude that it doesn't ever end up being worth it when there's no alternative that doesn't also produce some bugs.

I don't particularly like these tools, but I also can't deny that for some things I'm genuinely able to get to the results I want faster with them than without. Yes, some people will use them to produce low quality software, but in my experience it's mostly people who already would have been producing low quality software as well, just at a much slower pace. Having a higher volume of low quality software is a problem, but it's distinct from the claim that the tools not ever being useful in producing high quality software.

It's possible that from reading my rebuttals you'll think I'm one of the people using these tools to produce low quality software, but it's not clear how to falsify that claim. If that's the conclusion you'd draw, then you probably weren't really open to considering rebuttals in the first place though.

Re: An Honest Review of AI Programming

#52

Post by perching_aix »

Sharlin wrote: post #8
daishi55 wrote: post #3

> “Agentic workflow” (or whatever they’re calling it at the time you’re reading this article) > Unlike the silver bullets of the past (like microservices or NoSQL) > Hallucinations are an inherent property of how LLMs work. > It’s all marketing and buzzwords Not a serious article or thinker. I can get this stuff on Reddit if I want to read thrice-regurgitated cliches about AI.

I'd like to hear actual rebuttals to those quotes.

Rebuttals to what? Ill-faith rhetorics and circular judgement?

Points 1-2 were clearly cited not because of their factual content, but because they boil down to the author mocking the existence of industry trends, and writing them off whole (incl. this one). This is unsound, both because it doesn't actually follow by default, and because it violates the principle of suspending (prior) judgement: https://en.wikipedia.org/wiki/Suspension_of_judgment - it sets both the author and their readers up for a specific conclusion, rather than inspiring nonbias. Hopefully one does not need to explain why this is problematic?

It is further incredibly trite to bring up how AI is an ongoing trend (self-evident), and the fact that trends distort perception (also self-evident). Them fighting fire with fire and bringing their own gut instinct is not any more intellectually respectable.

Point 3 is completely unsupported and misleading. The author probably means that hallucinations are formally unavoidable, but that on its own doesn't carry much weight, and is not the same thing.

Point 4 is factually wrong. AI is an entire academic field with a half a century of history to its name, and this is trivial to learn. Clearly not why it was cited once again either however, but because it's blatantly ill faith too.

See also: https://en.wikipedia.org/wiki/Motivated_reasoning

Re: An Honest Review of AI Programming

#53

Post by unknownfuture »

CuriouslyC wrote: post #30
FrustratedMonky wrote: post #28

Earlier quoted context omitted.

Maybe you guys just shouldn't read blogs.

It'd be nice if blog authors categorized their posts into "old man shaking fist at clouds" vs "something I searched for on the internet, legitimately never found it, and decided to write it myself"

The blogger owes you nothing.

Re: An Honest Review of AI Programming

#54

Post by beepbooptheory »

daishi55 wrote: post #17
nesarkvechnep wrote: post #10

Earlier quoted context omitted.

So you basically agree but complain there are no new arguments against AI programming in the article?

No, I’m not sure where you are getting the idea that I agree with what I am referring to as cliches last relevant in 2024.

How can it be both irrelevant and cliché?

Re: An Honest Review of AI Programming

#55

Post by brazukadev »

dr0idattack wrote: post #15

I liked the two original experiments in the article, which in a microcosm gels with my 1+ year of deep agentic development. But I would like a whole article about his experience, and with which model (Opus? Fable? erg, Sonnet?) and effort he used. I've progressed in using latest Claude-kins and the GPTs as usually competent teammate/buddies, and generally know to sort out the fluff confidence with the realz (shoot, t…

I have been doing this (vibecoding) for as long as the term vibecoding was coined. The amount of people I have met that describe the process like you did and then proceed to show the most soulless vibecoded app is staggering.

Re: An Honest Review of AI Programming

#56

Post by ryanackley »

cronin101 wrote: post #49
ryanackley wrote: post #34

Earlier quoted context omitted.

His experience is completely plausible. He’s in a niche that requires highly performant code and most complex, highly performant games do nit have source available for models to train on. It’s a very common observation that the farther you stray from mainstream, the less effective the LLM models become.

Have you actually tried performance optimisation using an agent? With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc. High performance algorithms are quite well documented so it isn't unreasonable…

LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market.

Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples and experiments at face value.

Re: An Honest Review of AI Programming

#57

Post by brazukadev »

daishi55 wrote: post #9
Sharlin wrote: post #8

Earlier quoted context omitted.

I'd like to hear actual rebuttals to those quotes.

If the author can’t put in the effort to think of something original to say that hasn’t been said 1000x, why should I put the effort into rebutting cliches that were last relevant in 2024?

what's the point of making a ragebait comment if you didn't like the article? Whatever you think about the post, your reaction is much lower quality.

Re: An Honest Review of AI Programming

#58

Post by cronin101 »

ryanackley wrote: post #56
cronin101 wrote: post #49

Earlier quoted context omitted.

Have you actually tried performance optimisation using an agent? With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc. High performance algorithms are quite well documented so it isn't unreasonable…

LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market. Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples an…

I’ve specifically used Opus to diagnose and fix performance bottlenecks in parallel Rust code on multiple occasions (e.g improving NPS for a chess engine) and it works well.

I’ve done plenty of performance architecting in my day-job and rule #1 is generally “you can’t fix what you can’t see/measure”. I have a suspicion that many folks aren’t investing in letting AI actually introspect iterative execution via the appropriate harness, and are then acting surprised that it is no oracle.

Re: An Honest Review of AI Programming

#59

Post by jdw64 »

ryanackley wrote: post #56
cronin101 wrote: post #49

Earlier quoted context omitted.

Have you actually tried performance optimisation using an agent? With any programming language/framework that has quality profiling tooling (which is a prerequisite for most projects) I have had huge success with automated hotspot profiling where the LLM can propose theories, test the impact of fixes, convince you of which to pursue, etc. High performance algorithms are quite well documented so it isn't unreasonable…

LLMs work perfectly fine for me but I’m building web app equivalents of binder keepers, like most people. Essentially store data, display data. The novelty is in what/how we’re displaying. LLMs are owning this market. Unless you yourself are doing commercial game development, calling the author disingenuous puts your own pro-AI bias on display. It’s based on speculation. I prefer to take his very specific examples an…

To be honest, the optimization the author talks about isn't really high level optimization. And Claude's pattern actually calls for a more extensible design. Strictly speaking, the author's instruction could be considered incorrect. I'm not trying to dismiss the author.

If performance were truly critical, they wouldn't have been using Unity in the first place. They would have used Unreal, as mentioned earlier. And if they were sticking with Unity, they would have tried ECS.

Unity is fundamentally based on the template method pattern. The idea of pulling Update out and handling it in a single manager class is really more of a small scale indie game approach. It's a technique that scales very poorly.

In practice, there are many better optimization techniques for GameUpdateable. So I'm not sure why this particular example was used to demonstrate performance optimization.

Typically, you could use GameUpdateable with object pooling, which would be a safer approach. There are also many batching techniques available.

In other words, this isn't about performance. It's a technique used for small indie game development. By handling it directly through a manager, registration and removal no longer depend on the Unity framework and become manually scheduled by the user. This, in turn, means you have to handle many more edge cases, which creates additional work. This is a common pattern, sacrificing future extensibility for immediate performance gains.

It's a technique used in small indie games. Converting per frame Update callbacks into a central loop that iterates over all objects is where GameUpdateable would actually be a better choice. So rather than viewing this as an optimization for performance, it should be understood as a design choice made to make small games easier to manage.[1]

[1]https://docs.unity3d.com/Manual/events-per-frame-optimizatio...

Re: An Honest Review of AI Programming

#60

Post by brazukadev »

cronin101 wrote: post #32

At the risk of sharing an unpopular take, this reads a lot like someone who decided how they feel about LLM-driven engineering ~5 months ago and doesn’t seem to reflect the current reality. (Or they’re working in an organization with lower budgets and not cranking the frontier models of today) I fully agree about the cost/sustainability parts, but to suggest you can’t build a high quality coding/verifying/iterating l…

not much changed in the past 5 months if you have been doing agentic development. Claude Code actually is getting worse.
Post reply on HN