Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Eternal Sloptember

geohot.github.io

Post reply on HN
Print view View original

141–150 of 389 posts

Re: The Eternal Sloptember

#141

Post by SCdF »

So currently there are people who are buying grey market peptides[1], marked "not for human consumption" and injecting themselves with them based on dubious anecdotes and vibes, to make their skin clearer, build muscle mass, and so on.

Are they are all suddenly turning into zombies? No. Do they have any real idea what that is going to do to their body a few years down the line? Also no. Could it be catastrophic? Maybe!

I think about this when I think about how violently much of the industry has pivoted into AI being the primary generator of code in the last 6ish months. AI is the peptide, your codebase[2] is the body. Literally no one knows how maintainable this approach is, because there simply hasn't been enough time to find out. It could be fine. It could be a complete mess, with your entire engineering team falling asleep at the wheel, lulled into thinking they understand what is being built when they don't, completely impotent to fix or maintain it once the LLM is no longer able to.

[1] https://www.bbc.co.uk/news/articles/cdr268m5pxro

[2] Well, _their_ codebase. I've stopped doing it with my own personal codebases, unless I genuinely don't care about maintainability or longevity

Re: The Eternal Sloptember

#142

Post by CTDOCodebases »

dathanb82 wrote: post #59
andrewflnr wrote: post #31

Earlier quoted context omitted.

I wish people would stop comparing AI with cryptocurrency. The hype/perception was the only thing that was similar between them. The fundamental usefulness of the respective technologies are not comparable.

Two other similarities: they both rely on burning huge amounts of electricity, and have driven up costs of GPUs around the world.

The cognitive dissonance around this is astounding.

First of all define productive. Would someone using AI to build software at a startup which is likely to fail be considered productive? What if there is already similar software available that solves the same problems? What about the broad use of LLMs to draft emails or make silly memes?

It’s funny how everyone’s concerns around climate change just disappeared when they realised AI was useful to them.

Re: The Eternal Sloptember

#143

Post by swazzy »

cafkafk wrote: post #74

I think a lot of the problem with the current discourse is how black-and-white it is. Either you're a luddite or "ai pilled". In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, sometimes, it just gets you somewhere wrong. But it seems everyone is arguing about whether LLMs can be perfect software engineers in isolation running in a closet, and using that to say that LLMs do n…

I think that's geohot's point as well. They're advocating against being fully "ai pilled". Saying we should be using AI as a tool, not for being a luddite.

Re: The Eternal Sloptember

#144

Post by cheevly »

Alex_L_Wood wrote: post #91
blobbers wrote: post #41

Earlier quoted context omitted.

It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done. Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern. The LLM in this use case is perfect because almost every…

It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall. As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.

Agents are perfectly capable of learning. Why would the model need to learn? The harness and tooling are all that matter.

Re: The Eternal Sloptember

#145

Post by latentsea »

sodafountan wrote: post #101
DeathArrow wrote: post #79

Earlier quoted context omitted.

>It does as well as you care to coax into doing tbh. You have to invest heavily in harness engineering, and at least my experience has been that as you do that, the results improve. That is also my experience. When starting a project I observe how the agent fails, I add new rules to the harness to prevent it from falling and repeat the process until I am happy with the output.

I'm unfamiliar with harness engineering. Is there any good documentation about the subject you could point me to?

https://openai.com/index/harness-engineering/

https://www.anthropic.com/engineering/harness-design-long-ru...

https://www.anthropic.com/engineering/effective-harnesses-fo...

These were some of the first major articles on it. It's becoming a popular topic, so there's more content on it all the time.

Re: The Eternal Sloptember

#146

Post by shadowfacts »

thedjpetersen wrote: post #54

Part of my job is working on trying to make these models productive for the large corporation I work for. It's a lot of throwing tomatoes at a wall and to a degree I see the issue he is talking about output seemingly having a certain ceiling. At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of critici…

When people write blog posts about how LLMs failed for some particular task, the responses from boosters invariably fall along the lines of "just use this other model/just tweak your prompt like so/you're just not skilled enough—you can't make fundamental arguments about AI by citing specific examples."

So we can't make arguments by citing specific examples, and also can't make arguments by not citing specific examples. Whelp, I guess that's the ball game.

(yes yes, I'm committing a group attribution error, but still)

Re: The Eternal Sloptember

#147

Post by luodaint »

Data from six months of production from one SaaS codebase provides a more limited response. Maintainability doesn't depend on the level of AI usage. Maintainability depends on the discipline during diff reviews. Good sessions: One topic per session; scope defined prior to the agent starting; all diffs read prior to committing. Poor sessions: Broad scope; undefined constraints; rubber-stamped results.

The quality of the codebase decays precisely at the rate you stop reading the results. This is not an issue of AI writing the code. This is an issue of unreviewed code. geohot's issue is entirely valid. This problem does exist. But this isn't dependent on the generation phase.

Re: The Eternal Sloptember

#148

Post by latentsea »

lazide wrote: post #134
latentsea wrote: post #67

Earlier quoted context omitted.

> Yes, I don't have anything important to say other than I 100% agree with this comment. AI in its current state is akin to Stack Overflow and Google on steroids, but from my experience, it doesn't do well building out full-scale applications other than perhaps some initial scaffolding. We're currently using it to build out a full-scale application. It does as well as you care to coax into doing tbh. You have to inve…

In my experience, you’ll eventually hit a context window issue and it will just start spouting gibberish/doing wrong things, and nothing will significantly improve it. But hey, maybe it’s improved.

Well, auto-compaction is a thing in Claude Code now. Plus we have /goal command and some automated review stuff, so you can kinda just get it to loop until the automated reviews are satisfied and CI is passing. Does most of the heavy lifting.

Re: The Eternal Sloptember

#149

Post by xarope »

simianwords wrote: post #72
vips7L wrote: post #62

Earlier quoted context omitted.

If you don’t know what library to use in your specific language, do you think you know enough to have an LLM generate most of it?

YES. This line of thought is exactly why people are still skeptical of LLM's. LLM's are directionally right and if their answer "fits" then I take it at face value. I wrote a blog detailing the computational difference between "generation" and "verification" and why it matters for LLM's: https://simianwords.bearblog.dev/the-generation-vs-verificat... As an example: I asked the LLM "synonym for "provides" that also me…

that's a very dangerous analogy, because you would be considered the domain expert and you are just asking for synonyms for something you already know but may not remember off-hand.

now, what if you asked for the synonym for "provides" in a language that has gender differences (e.g. spanish/portuguese) as well as societal nuances (e.g. japanese) and it gives you "confers", how would you now know that's correct?

ah, so you say you tell it to take into consideration gender differences, as well as societal nuances. What are those, if you were not already familiar with the language?

Re: The Eternal Sloptember

#150

Post by ben_w »

tonyedgecombe wrote: post #110
cafkafk wrote: post #74

I think a lot of the problem with the current discourse is how black-and-white it is. Either you're a luddite or "ai pilled". In most cases, LLMs can get you 80-95% of the way, sometimes less, sometimes more. And heck, sometimes, it just gets you somewhere wrong. But it seems everyone is arguing about whether LLMs can be perfect software engineers in isolation running in a closet, and using that to say that LLMs do n…

>I think a lot of the problem with the current discourse is how black-and-white it is. There is too much money involved for any rational debate.

Only on the pro-AI side. The "it is bad" side is diverse on the reasons why, but being overwhelmed by bad content isn't a monetary concern.
Post reply on HN