Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Eternal Sloptember

geohot.github.io

Post reply on HN
Print view View original

91–100 of 389 posts

Re: The Eternal Sloptember

#91

Post by Alex_L_Wood »

blobbers wrote: post #41
Alex_L_Wood wrote: post #9

Earlier quoted context omitted.

Well, since the fundamental underlying structure is still the same, yes.

It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done. Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern. The LLM in this use case is perfect because almost every…

It's not just that, but the core is just that, even with reasoning models. Harness can only get you closer to the good result, but can't save you from every pitfall. As for PM analogy - don't forget that models don't learn and keep doing same stupid stuff they were doing a month ago.

Re: The Eternal Sloptember

#92

Post by georgehotz »

wyager wrote: post #5

> They are a highly sophisticated statistical model designed to mimic the distribution of programming Are we really still doing this?

Author here. I have never said that phrase before this blog post and certainly understand the absurdity of it. I certainly don't mean that you need something biological or whatever consciousness might or might not be.

However there's still a distinction. Unless I'm responding to an LLM, you had a childhood. You learned about the world and space and agency before you ever learned how to program. And you didn't learn it from billions of examples, you learned from a few examples, some self directed experiments, some feedback from teachers, etc...

I'm saying that's what matters. The process matters. You didn't learn to mimic a distribution, you learned to program. Of course in the perfect mathematical limit it's the same, but in practice it's not.

Re: The Eternal Sloptember

#93

Post by maxnevermind »

farhanhubble wrote: post #14

I'm in the "haven't written any code in a while" boat ATM. I'd love to see examples of issues that are so big that they warrant reverting to manual coding. My main issue has been the inconsistent quality across between model releases and the tendency to insert older APIs or documentation, especially with command line tools. I can understand if the model struggles with a million line monolithic codebase with a decade…

What type of projects you work on, in particular how rich it is in novelty, non-googlable data points and non-trivial project-specific deviations from industry standards?

Re: The Eternal Sloptember

#95

Post by theojulienne »

dathanb82 wrote: post #53
protocolture wrote: post #42

Eh but statistical models are obviously useful, because statistically 99% of your codebase wont involve new idea invention. Tools that write all the boilerplate code used to have names and job titles. I hate how both the for and against case for LLMs are just so bloody terrible at addressing these things.

This is a good take. The most effective combination of AI and skilled practitioner is using AI to amplify the abilities of the skilled practitioner. And in particular, max benefit comes from exploiting comparative advantage. AIs are really good at boilerplate -- in many cases better than humans because humans will optimize the process by doing copy/paste and often inject errors in the process -- whereas humans are be…

This. A ~staff software engineer designing big changes at one level above the raw implementation details using Opus 4.7 + superpowers today can genuinely ship multiple times more at the same quality level than pre-AI. The level of what a whole team could ship before.

You have to use something like superpowers, the key is that the humans need to make the important decisions.

You have to review the code - just like you had to review the code humans wrote. There will be iterations.

You have to give the LLM skills and patterns to follow, access to architectural documents, etc, just like humans needed to be onboarded at a company and do the same.

If you get all of these right with today's LLMs, you will never write code at all because it is so obviously not the best use of your time. If you feel that you are still better at writing the code manually, you have not done the above right, fix your workflow and try again.

Re: The Eternal Sloptember

#96

Post by siriusastrebe »

bluegatty wrote: post #52
nilirl wrote: post #38

I agree that I can write better code than an agent. But it can write working code much faster than I can. And in a lot of cases, unfortunately, faster beats better.

At a granular level, it's almost guaranteed that you cannot write better code than an agent. Agents now are writing extremely consistent, normalized canonical code, that usually compiles the first time. Right out of the 'textbook'. For what it's trying to do - it writes nearly perfect code. The only thing you could nominally disagree with are some of the conventions and idioms. It 'writes a perfect novel, in perfect…

I don't think LLMs inherently do anything perfectly. They can make sure it compiles and passes tests and they can be trained to do an enormous array of tasks, but the code it generates isn't perfect, it's selecting one of many possible outputs based off of some numbers it came up with after a few matrix multiplications and ReLU activations.

Those matrix multiplications aren't a divine perfect thing. They suffer from floating point precision issues and training data issues and there's still debate if adversarial examples are just an unsolveable property of our linear-algebra based neural network architecture.

Can they do things way faster than a human? No doubt. Can they do very complex tasks? Yes. Do they do things with perfection? Not by our human definition of perfect.

Re: The Eternal Sloptember

#97

Post by Swizec »

thedjpetersen wrote: post #54

Part of my job is working on trying to make these models productive for the large corporation I work for. It's a lot of throwing tomatoes at a wall and to a degree I see the issue he is talking about output seemingly having a certain ceiling. At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of critici…

> How are we really supposed to grasp their actual capabilities when no one will actually cite specifically what mistakes they are making.

The mistakes they make are pretty subtle. Coding with LLMs can be like that scene in Whiplash – , not quite my tempo, , downbeat on 18, , you’re rushing, , dragging, …

Like yeah it produces working code almost always and the code usually does what you asked. And yet it makes you want to throw a chair because it’s not quite right in frustrating ways and it doesn’t even have the taste to know how it’s wrong.

Re: The Eternal Sloptember

#98

Post by asixicle »

thedjpetersen wrote: post #54

Part of my job is working on trying to make these models productive for the large corporation I work for. It's a lot of throwing tomatoes at a wall and to a degree I see the issue he is talking about output seemingly having a certain ceiling. At the same time in no part of his post is any code snippet or anything to latch on to of "the model performed poorly here when it should have done this" - this style of critici…

This is an excellent point, and as a novice using LLMs for projects I could never previously dream of doing I find myself looking for the same, examples or citations of what exactly agents are writing incorrectly and how would the human do it better. I'm sure they're out there, maybe someone can refer some good content showing such examples.

I have no doubt the top nth percent of coders could write circles around Claude or Codex, but how much worse are they than your average schnook?

Re: The Eternal Sloptember

#99

Post by kelnos »

blobbers wrote: post #41
Alex_L_Wood wrote: post #9

Earlier quoted context omitted.

Well, since the fundamental underlying structure is still the same, yes.

It's not exactly what it is; they now model an incredibly complex markov process, and harnesses that control how that thinking is done. Is this any different than how a PM gets a programmer to work on a project? They think, then they deliver. If given more time, maybe they deliver something better. Maybe they consult some text and try to apply a design pattern. The LLM in this use case is perfect because almost every…

> Is this any different than how a PM gets a programmer to work on a project?

Yes, it's very different. You seem to be suggesting that the current frontier LLMs, when tied to their tools and harnesses, have emergent properties that are similar to human consciousness. If you truly believe that, I'm not sure how to have a productive discussion here.

Re: The Eternal Sloptember

#100

Post by albinn »

> It’s definitely a better Google for most searches

I can't agree with this. You tend to get one point of view, often without any actual resources and references so you have to go look it up yourself, on [insert search engine]. Plus, what does it say when we consider an AI the one stop for our data intakes.

Post reply on HN