Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Eternal Sloptember

geohot.github.io

Post reply on HN
Print view View original

61–70 of 389 posts

Re: The Eternal Sloptember

#61

Post by DeathArrow »

To me this sounds like an old cobbler complaining that machines aren't producing good shoes if left unsupervised and that the old process of making shoes completely by hand is far superior.

So what he is telling us? That agents are not infaillable and they are not capable to one shot complex software and they do not produce perfect code?

We know what and the solution is to use agents for what they are good at and work around their limitations and we have a human in the loop.

>not some RLVR shit that comments out the failing test and tells you all the tests are now passing

That's what harnesses should be about: detect when the agent is misbehaving and force it to take the right approach.

This example in particular should be easy to solve if we generated the tests before coding and we have a workflow or state machine that doesn't allow the agent to disable tests and doesn't allow it to reach the next stage unless all tests are passing.

Re: The Eternal Sloptember

#62

Post by vips7L »

simianwords wrote: post #32

People misunderstand how AI is used in coding in normal work environments. New feature requirement comes - maybe you need a new service or some new classes. You need to do some research first. You guide the AI with some prompts and give it some guidance on how to scenario-test it. It makes some classes, test methods. Maybe ~2000 lines and you do a quick verification, check if the overall idea looks okay. Ask it to fi…

If you don’t know what library to use in your specific language, do you think you know enough to have an LLM generate most of it?

Re: The Eternal Sloptember

#64

Post by slashdave »

> It is a golden era for buckets and buckets of slop, and a dark age for gems of quality.

I mean, this has been the trend for decades really, before LLMs were a thing. The incentive is skewed toward quantity rather than quality. The new tools just add more fuel to the fire.

Code quality is also really lacking in much of the industry. The truth is, these LLM models, as limited as they are, program at a level above that of the median junior programmer.

Re: The Eternal Sloptember

#65

Post by mrec »

Chaosvex wrote: post #44

> and it’s taking longer and longer to realize that they can’t For something to take "longer and longer" to realise, doesn't they imply that it's been realised at least once before or that there was an expected deadline for the realisation? Okay, that's a nitpick.

I read it as "agents can't program, and with each new generation of agents it's taking longer and longer to realize that that specific iteration can't". Maybe taking the Principle of Charity too far, I dunno.

Re: The Eternal Sloptember

#66

Post by webprofusion »

I don't think you can go completely hands-off for quality products but you can relax and let the agent do as much as possible. It does enable things that probably wouldn't have happened otherwise.

If you are already comfortable with letting other devs work on features then it's easier, because it's similar (arguably you have more control with AI, because what you say goes regardless of hierarchy).

Re: The Eternal Sloptember

#67

Post by latentsea »

sodafountan wrote: post #49
Nition wrote: post #17

With the level of ability that AI is at right now, I've found it useful personally to think of it something like a very good search over existing knowledge. Another step up in searchability in the lineage of reference books, stack overflow, GitHub etc. Programmers are rewriting and reinventing the same techniques more often than any other vocation I can think of, and so we were primed for a really good search over pr…

Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding. If I were to use it against a legacy, rather poorly written codebase, where the code may be hard to understand without some in-depth a…

> Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding.

We're currently using it to build out a full-scale application. It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve.

Re: The Eternal Sloptember

#68

Post by hu3 »

bandrami wrote: post #12
mountainriver wrote: post #4

My guess is the models just continue to get better and better When I got into agentic coding a year or two ago I was sure it was only good at autocomplete. Something happened earlier this year where the models hit a new level of capability. Everyone I know now just does agentic coding, and it’s really amazing. I think we should just try pushing this as far as we can possibly go, it really feels like the acceleration…

We're already hitting some logistical limits. Even if transformers don't have an inherent capability plateau, we only have so many GPUs and so much power to improve them, and we're finding it very difficult to expand that infrastructure. Something like 6 GW of new DCs have been announced over the past 2 years of which less than 1 GW has actually been turned on and started serving, and the deliverable dates for the re…

Sounds like we just need a Dyson sphere.

Besides, I have been hearing "this is the limit" since the doomers of "this is just a markov chain and can't be useful".

Yet the limits keep being broken.

Re: The Eternal Sloptember

#69

Post by baq »

Wonder if LLMs in autoreasearch loops would be able to complete tasks geohot has in mind in say 100x average token budget.

If the answer is yes, the argument doesn’t matter: you just run the loop and wait for llm analog of moore’s law to get costs down.

Re: The Eternal Sloptember

#70

Post by HerbManic »

nilirl wrote: post #38

I agree that I can write better code than an agent. But it can write working code much faster than I can. And in a lot of cases, unfortunately, faster beats better.

> Unfortunately, faster beats better.

I think you have just written the epitaph for corporate software.

Post reply on HN