Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Coming Loop

lucumr.pocoo.org

Post reply on HN
Print view View original

301–310 of 322 posts

Re: The Coming Loop

#301

Post by Aperocky »

dofm wrote: post #96
Aperocky wrote: post #92

Earlier quoted context omitted.

I think at the end of this we'll have a new software engineering paradigm. Mostly nobody now worry about binaries or instructions because those are for the compilers, even undefined behavior are mostly ignored. You can either tailor the development pattern for LLM, or have LLM come write for the same old development pattern. I think there is going to be a difference.

We will have a new software engineering paradigm and it is called quitting to become a day drinker. Burnout can be so all-encompassing that it can ruin your life. It can make you feel like you are losing your mind and your basic abilities. A lot of that is coming for a lot of people who are currently stressed but excited.

That's a natural result of introducing LLM to current standard software development without modifying the rest of it (the part that doesn't involve creating code).

Re: The Coming Loop

#302

Post by zahlman »

simonreiff wrote: post #142

AI infrastructure/tools developer and researcher here (hic-ai.com). I fully agree with Armin's concerns. I wrote an article recently ( https://hic-ai.com/blog/tool-response-engineering ) in which I argued that AI tool-engineering is the new frontier beyond prompts, and it talks about the agent loop and engineering loops, but boy I have a completely different perspective than Boris's. Rather than contending that promp…

> AI agents lack durable, long-lasting, unique identities forged by real-world memories and experience, and they therefore lack judgment.

I agree with all of this, but I think there's also a simple information theory issue. If the only actual inputs to the system are one prompt and a PRNG stream....

Re: The Coming Loop

#303

Post by TehShrike »

sixhobbits wrote: post #179

I have huge respect for Armin but all of the concerns about agents producing more code with less competent supervision from senior engineers doesn't seem that different from the status quo to me. A vast majority of all software I've ever professionally worked with has been terribly structured, hard to work with, full of bugs, etc, produced by mediocre to bad engineers and run by semi technical product owners or manag…

Yes, a lot of software shops have produced equivalent slop, but I've worked on teams that aimed higher, and they were much more productive over the long haul – not just in fewer bugs, but also in straight up ability to add more features as the software got complicated.

Re: The Coming Loop

#304

Post by soulofmischief »

bee_rider wrote: post #218
soulofmischief wrote: post #153

Earlier quoted context omitted.

Billion dollar one person startups are a silly target too, because as we approach that possibility, society will adjust. I know Sam Altman peddled this concept, but he's not representative of the culture and principles of the entire AI/ML space. So it was wrong of OP to attempt to use his words against the entire idea of neural models.

I do think the point that solo companies that could have been billion dollar start-ups might get a lower market cap if they can be done solo with AI tools is a good one. But that seems a lot more nuanced than calling the idea something like a Mach 1 car. I think this is like pointing out that we lowered the speed of sound as technology developed, so a Mach 1 car doesn’t mean the same thing anymore, if we want the ana…

You have a nuanced view which I'm taking into consideration. I agree that if I'd spent any amount of time on my comment, I might have found a more apt analogy.

The OP's response, "It is a terrible analogy that shows terrible thinking," is however entirely ridiculous and taking my comment in bad faith, against hacker news guidelines.

As such, I do contend the analogy fits, even if it doesn't fit perfectly.

Re: The Coming Loop

#305

Post by archagon »

meowface wrote: post #288
archagon wrote: post #148

Earlier quoted context omitted.

What a smug, dickish way to try to make a point. (But he’s still on Twitter, so I guess that makes sense.)

I detest Elon Musk but I still use Twitter because there just is no alternative to it if you want decent tech discussion. And I don't think it's smug or dickish. I like AI and I encourage even inexperienced software developers to use it. There's just no denying that the slop cannon phenomenon is still a very real phenomenon.

I’m thinking more of the “Dead Weight” and “Turbo Brains” boxes as being particularly eye-rolling.

As for tech discussion, of course there is an alternative. Practically every tech account I used to follow has migrated to Bluesky or Mastodon by now. The accounts left on Twitter have a distinctly narcissistic, preachy vibe. These people care more about “influencing,” not engaging. (And obviously they have no problem with the new ownership.)

Re: The Coming Loop

#306

Post by casey2 »

The most important part of working with agents is removing tools, especially tools they like to use. Do not let them use cc, make, git etc until it's a necessity 90% of changes introduce a bug? ok you can start using gcc/clang/rustc whatever. Tools generate too many tokens, and it can't be fixed by '| grep' since they just start spamming cli calls.

At least this is true for antigravity it tried to used git a few times and I gave in and then I discovered there was a token limit. Velocity plummeted. If you want git before it becomes a requirement best to mirror the directory into a separate git repo.

Re: The Coming Loop

#307

Post by watutalkinbout »

coldtea wrote: post #184
watutalkinbout wrote: post #171

Earlier quoted context omitted.

Algorithms and data that emulate responses aren't smart. A 5 year old knows if you want to wash your car, you need to take it to the car wash.

Can a 5 year old write a substantial program on spec, that passes the requirements and given tests, in a few minutes? If not, then perhaps this comparison is not the be all end all. "A ship is useless, it can't drive over land..."

You're describing a tool.

Tools can do very useful things, but they aren't intelligent.

Re: The Coming Loop

#308

Post by watutalkinbout »

Eisenstein wrote: post #293
JBits wrote: post #194

Earlier quoted context omitted.

But it demonstrates that LLMs struggle with basic reasoning. A criticism of LLMs is that they're imitating without a understanding of what they're doing and without a clear plan, so this inability to solve a simple logic puzzle is very relevant. If LLMs didn't struggle with reasoning problems then something like ARC-AGI wouldn't exist.

Its a question designed to fool the AI. It's like saying that a person doesn't understand the limitations of reality when they fall for a magic trick.

These aren't the same thing.

You can't fool an AI because it isn't using its own judgement.

Re: The Coming Loop

#309

Post by citizenpaul »

jongjong wrote: post #259
gavinh wrote: post #248

> We may create codebases that are not merely hard to maintain by humans, but that assume machine participation as part of their maintenance model... People more and more merge code they cannot fully explain. People lose their ability to create issue reports or discuss things in chat, without augmenting or rephrasing their messages with the context provided by a clanker. Too many people increasingly rely on a machine…

And this code is often full of security vulnerabilities. It's just hacks on top of hacks on top of hacks. You end up with 100K lines of code full of weird fallbacks, doing something which could have been done more reliably with just 1K lines of code. I think author's comment about preferring systems which make invalid edge cases impossible rather than implementing fallbacks is hugely important. With the fallback appr…

My experience with LLM's is that the "fallback" issue is probably one of the most serious issues. I have seen zero talk about it outside of this thread. I'm not sure its even being worked on. The llms have this terrible drift towards always making something happen even if that thing is not even related to the task at hand. "failure" in the sense of simply throwing an error/exception is something models seem highly resistant to.

I cant tell you at this point how many times I've seen them do something like

Fail > make up values > maybe log it > keep working silently with increasingly corrupt data.

Re: The Coming Loop

#310

Post by bonigv »

This is a fascinating thought. I agree. It is one thing to let it write a prompt. Prompting, Generating, Judging and going till it is satiated is gluttony.

Every week I ship 4-5 sprints' worth of work that improves features, simplifies code, adds maintainability while delivering value to my end users without feeling I butchered a software hand-engineered over 10 years. This is one product. Mentioning it as an example because it got built over the years starting 2010. It has ~7000 paying customers and works in the B2B space. The B2B SaaS space is brutal since the defendable value should be delivered with margins two levels deep. I wouldn't be able to do it w/o coding agents.

BUT I will never do it with LOOP ENGINEERING. Not in 2026 and not in 2036. Software Engineering as such is not going to be much different in 2036 than it is now.

What stops me is the accountability that lies with me in delivered software. Setting lofty goals, a few guardrails and then letting harness and agent rip it overnight can (and have) created slop.

I did not find LLMs systematically generate poor code. What is poor? Poor in traditional handcraft coding days is defined by

- Poor performance - Verbosity - Complexity - Hardcoding - Poor maintainability

These are guardrails that the models understand. However it cannot understand all these upfront - when building new. It always needs a retrospection cycle and a token budget to refactor.

This is tech debt which we all carried and solved. It is just at an exponential scale because - well, the generation is happening at a super human scale.

So, either build the harness with the tech debt coverage cycles or take on ourselves to add that cycle triggered by human judgement.

A few things that are working for me and I trust are:

Forge CLI - the harness I use with a built-in engineering philosophy for everyday work [https://github.com/Entelligentsia/forge-cli]

A few patterns that I have known to trust for building software that can be maintained - expressed as skills [https://github.com/Entelligentsia/skillforge/tree/main/desig...]

A few patterns for the new probabilistic constructs we build with in 2026 LLM Patterns [https://github.com/Entelligentsia/skillforge/tree/main/desig...] Harness Patterns [https://github.com/Entelligentsia/skillforge/tree/main/harne...]

The harness gives me a process. Patterns give me architectural guardrails as I could enforce (with a prayer). But still read code. We gotta.

Post reply on HN