Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Coding agents have replaced every framework I used

blog.alaindichiappari.dev

Post reply on HN
Print view View original

341–350 of 611 posts

Re: Coding agents have replaced every framework I used

#341

Post by scoofy »

robhlt wrote: post #333
mark242 wrote: post #215

Earlier quoted context omitted.

I would argue that it's going to be the opposite. At re:Invent, one of the popular sessions was in creating a trio of SRE agents, one of which did nothing but read logs and report errors, one of which did analysis of the errors and triaged and proposed fixes, and one to do the work and submit PRs to your repo. Then, as part of the session, you would artificially introduce a bug into the system, then run into the bug…

It's nice that AI can fix bugs fast, but it's better to not even have bugs in the first place. By using someone else's battle tested code (like a framework) you can at least avoid the bugs they've already encountered and fixed.

I spent Dry January working on a new coding project and since all my nerd friends have been telling me to try to code with LLM's I gave it a shot and signed up to Google Gemini...

All I can say is "holy shit, I'm a believer." I've probably got close to a year's worth of coding done in a month and a half.

Busy work that would have taken me a day to look up, figure out, and write -- boring shit like matplotlib illustrations -- they are trivial now.

Things that are ideas that I'm not sure how to implement "what are some different ways to do this weird thing" that I would have spend a week on trying to figure out a reasonable approach, no, it's basically got two or three decent ideas right away, even if they're not perfect. There was one vectorization approach I would have never thought of that I'm now using.

Is the LLM wrong? Yes, all the damn time! Do I need to, you know, actually do a code review then I'm implementing ideas? Very much yes! Do I get into a back and forth battle with the LLM when it gets starts spitting out nonsense, shut the chat down, and start over with a newly primed window? Yes, about once every couple of days.

It's still absolutely incredible. I've been a skeptic for a very long time. I studied philosophy, and the conceptions people have of language and Truth get completely garbled by an LLM that isn't really a mind that can think in the way we do. That said, holy shit it can do an absolute ton of busy work.

Re: Coding agents have replaced every framework I used

#342

Post by MrDarcy »

robhlt wrote: post #333
mark242 wrote: post #215

Earlier quoted context omitted.

I would argue that it's going to be the opposite. At re:Invent, one of the popular sessions was in creating a trio of SRE agents, one of which did nothing but read logs and report errors, one of which did analysis of the errors and triaged and proposed fixes, and one to do the work and submit PRs to your repo. Then, as part of the session, you would artificially introduce a bug into the system, then run into the bug…

It's nice that AI can fix bugs fast, but it's better to not even have bugs in the first place. By using someone else's battle tested code (like a framework) you can at least avoid the bugs they've already encountered and fixed.

In practice using someone else’s framework means you’re accepting the risk of the thousands of bugs in the framework that have no relevance to your business use case and will never be fixed.

Re: Coding agents have replaced every framework I used

#343

Post by catlifeonmars »

tappio wrote: post #307
catlifeonmars wrote: post #280

Earlier quoted context omitted.

LLMs famously aren’t that good at using new frameworks/languages. Sure they can get by with the right context, but most people are pointing them at standard frameworks in common languages to maximize the quality of their output.

This is not my experience any longer. With properly set feedback loop and frameworks documentation it does not seem to matter much if they are working with completely novel stuff or not. Of course, when that is not available they hallucinate, but who anymore does that even? Anyone can see that LLMs are just glorified auto-complete machines, so you really have to put a lot of work in the enviroment they operate and qu…

Or you could use an off the shelf popular framework in Python and save yourself some time curating the context.

Re: Coding agents have replaced every framework I used

#344

Post by mrinterweb »

This article has some cowboy coding themes I don't agree with. If the takeaway from the article is that frameworks are bad for the age of AI, I would disagree with that. Standardization, and working with a team of developers all using the same framework has huge benefits. The same is true with agents. Agents have finite context, when an agent knows it is using rails, it automatically can assume a lot about how things work. LLM training data has a lot of framework use patterns deeply instilled. Agents using frameworks that LLMs have extensive training on produce high quality, consistent results without needing to provide a bunch of custom context for bespoke foundational code. Multiple devs and agents all using a well known framework automatically benefit from a shared mental model.

When there are multiple devs + agents all interacting with the same code base, consistency and standards are essential for maintainability. Each time a dev fires up their agent for a framework their context doesn't need to be saturated with bespoke foundational information. LLM and devs can leverage their extensive training when using a framework.

I didn't even touch on all the other benefits mature frameworks bring outside of shared mental model: security hardening, teams providing security patches, performance tuning, dependability, documentation, 3rd party ecosystems. etc.

Re: Coding agents have replaced every framework I used

#345

Post by oxag3n »

No, it copied some relevant parts of every framework code, shifting burden of design, maintenance, debugging and polishing all corner cases to your shoulders.

Why wouldn't you clone frameworks code into your repository, removing parts you don't need, modifying code as you wish?

Re: Coding agents have replaced every framework I used

#346

Post by hakunin »

philipswood wrote: post #262
hakunin wrote: post #129

Earlier quoted context omitted.

It’s not _more_ deterministic. It’s deterministic, period. The LLMs we use today are simply not.

Build systems may be deterministic in the narrow sense you use, but significant extra effort is required to make them reproducible. Engineering in the broader sense often deals with managing the outputs of variable systems to get known good outcomes to acceptable tolerances. Edit: added second paragraph

I'm not using a narrow sense. There is no elasticity here. See https://en.wikipedia.org/wiki/Deterministic_system

> significant extra effort is required to make them reproducible.

Zero extra effort is required. It is reproducible. The same input produces the same output. The "my machine" in "Works on my machine" is an example of input.

> Engineering in the broader sense often deals with managing the outputs of variable systems to get known good outcomes to acceptable tolerances.

You can have unreliable AIs building a thing, with some guidance and self-course-correction. What you can't have is outcomes also verified by unreliable AIs who may be prompt-injected to say "looks good". You can't do unreliable _everything_: planning, execution, verification.

If an AI decided to code an AI-bound implementation, then even tolerance verification could be completely out of whack. Your system could pass today and fail tomorrow. It's layers and layers of moving ground. You have to put the stake down somewhere. For software, I say it has to be code. Otherwise, AI shouldn't build software, it should replace it.

That said, you can build seemingly working things on moving ground, that bring value. It's a brave new world. We're yet to see if we're heading for net gain or net loss.

Re: Coding agents have replaced every framework I used

#347

Post by manuelabeledo »

rglover wrote: post #139

A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future. You can build things this way, and they may work for a time, but you don't know what you don't know (and experience teaches you that you only find most stuff by building/struggling; not sipping a soda while the AI blurts out potentially secure/stable code). The hubris around AI is goi…

There are people out there who truly believe that they can outsource the building of highly complex systems by politely asking a machine, and ultimately will end up tasking the same machine to tell them how these systems should be built.

Now, if I were in business with any of these people, why would I be paying them hundreds of thousands, plus the hundreds of thousands in LLM subscriptions they need to barely function, when they cannot produce a single valuable thought?

Re: Coding agents have replaced every framework I used

#348

Post by thwarted »

jack_pp wrote: post #134
thwarted wrote: post #94

Earlier quoted context omitted.

I tend to believe that, in all things, the quality of the output and how it is received is what matters and not the process that leads to producing the output. If the ends justifies the means is a well-worn disagreement/debate, and I think the only solid conclusion we've come to as a society is that it depends .

That's a moral debate, not suitable for this discussion. The discussion at hand is about purity and efficiency. Some people are process oriented, perfectionists, purists that take great pride in how they made something. Even if the thing they made isn't useful at all to anyone except to stroke their own ego. Others are more practical and see a tool as a tool, not every hammer you make needs to be beautiful and made f…

It sure is myopic to think that the debate over if the ends justifies the means is solely a moral consideration, and then literally list cases where the value of the means compared to the ends is a judgment call results in "it depends".

Re: Coding agents have replaced every framework I used

#349

Post by 113 »

mark242 wrote: post #215
rglover wrote: post #139

A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future. You can build things this way, and they may work for a time, but you don't know what you don't know (and experience teaches you that you only find most stuff by building/struggling; not sipping a soda while the AI blurts out potentially secure/stable code). The hubris around AI is goi…

I would argue that it's going to be the opposite. At re:Invent, one of the popular sessions was in creating a trio of SRE agents, one of which did nothing but read logs and report errors, one of which did analysis of the errors and triaged and proposed fixes, and one to do the work and submit PRs to your repo. Then, as part of the session, you would artificially introduce a bug into the system, then run into the bug…

> one to do the work and submit PRs to your repo

Have we not seen loads of examples of terrible AI generated RPs every week on this site?

Re: Coding agents have replaced every framework I used

#350

Post by kaydub »

bthornbury wrote: post #244
rglover wrote: post #139

A significant number of developers and businesses are going to have an absolutely brutal rude awakening in the not too distant future. You can build things this way, and they may work for a time, but you don't know what you don't know (and experience teaches you that you only find most stuff by building/struggling; not sipping a soda while the AI blurts out potentially secure/stable code). The hubris around AI is goi…

Why does there seem to be such a divide in opinions on AI in coding? Meanwhile those who "get it" have been improving their productivity for literally years now.

There's a lot of reasons. There's a lot of breadth to "software engineering" (FAANG, web dev, embedded, OS, small business, etc.)

I'm sure there are some places where LLMs are bad due to lack of training data. There are some places where the LLMs are bad because the code base is terrible (and there's always "rockstars" at these jobs that severely overestimate their skills because they're always the one fixing the mess... which they also probably caused). Some devs/engineers feel threatened. Many devs/engineers think they're special and super smart so surely no machine can do their job.

Post reply on HN