This is such a poor idea. LLM Agents don't have taste (abstraction capabilities). They are kind of like water, they take the shape of the container they're in. They are so much powerful when you build your own framework they can work in.
Retire the Abstractions
11–20 of 61 posts
Re: Retire the Abstractions
#12it sounds like a bad abstraction. ime llms are really bad at reasoning through efficiency and utilization problems. they fail at scheduling and bin-packing for the same reason. it's actually difficult, multivariate, and not great. i would be shocked if a well designed framework were less efficient than "bespoke" clankercode for low-level work
I mean, the post was written by a Stanford CS lab, so I'm inclined to believe that they know what they're talking about and are not just bad at creating abstractions.
The machine learning community decided to choose Python as the main language, which costs probably tens of millions of dollars in bugs every year due to it not being type checked (astral sh is fixing this though)
Re: Retire the Abstractions
#13Earlier quoted context omitted.
I mean, the post was written by a Stanford CS lab, so I'm inclined to believe that they know what they're talking about and are not just bad at creating abstractions.
why?
I think it's generally fair to assume that you don't become a Stanford CS professor by being bad at writing code and creating abstractions, and that the average professor (let alone one at a prestigious university) is more knowledgable than your average HN commentor
Re: Retire the Abstractions
#14Earlier quoted context omitted.
I mean, the post was written by a Stanford CS lab, so I'm inclined to believe that they know what they're talking about and are not just bad at creating abstractions.
I know nothing about the PhD candidate and professor who co-authored this, but I don't think this statement makes sense in general. Typically CS labs at universities contain people who have not spent a significant amount of time exposed to large industry codebases and the corresponding complexity. I think the post would have more credibility coming from e.g. the platforms team at a tech company with a monorepo.
There is something to be argued about industry vs academic experience but this post has nothing to do with large industry codebases
Re: Retire the Abstractions
#15agents are unlike humans in many ways but they have their own limitations. it's magical thinking to believe that they are able to cope with zero software abstractions, especially as a codebase increases in size and complexity and today's agents increase codebase size and complexity much faster than humans do via traditional software engineering methods, so this is likely to hit a wall faster than if human software te…
Just yesterday, for fun, I asked Codex to generate an “online e-commerce site” in ARM assembly. To its credit, it did actually generate a program that served up a webpage over a socket, but it really did not do what I asked. The program still pulled in a large number of shared libraries (something I’ve never really thought to try in assembly). And… it was not actually a functioning e-commerce site. Iterating on that initial prompt proved painful—the program was extremely brittle and now I had to review and talk about ASSEMBLY. Maybe we can dispense with abstractions someday, but holy cow, not yet. I would much rather write an e-commerce platform in C than try to vibe-assembly my way to one.
Re: Retire the Abstractions
#16“First, an abstraction isn't just a way to manage cognitive load; it's a shared surface where application, reuse, and review attach.”
That is some seriously convoluted writing.
Re: Retire the Abstractions
#17Earlier quoted context omitted.
I mean, the post was written by a Stanford CS lab, so I'm inclined to believe that they know what they're talking about and are not just bad at creating abstractions.
I know nothing about the PhD candidate and professor who co-authored this, but I don't think this statement makes sense in general. Typically CS labs at universities contain people who have not spent a significant amount of time exposed to large industry codebases and the corresponding complexity. I think the post would have more credibility coming from e.g. the platforms team at a tech company with a monorepo.
sounds like it might help to know something about the phd candidate and professor
Re: Retire the Abstractions
#18Precise specs must always be given, they cannot be reasoned into existence, because they are shaped by external considerations.
No matter how intelligent the executor is, without knowledge of the relevant external context, specs will always have to be clearly defined and given.
Re: Retire the Abstractions
#19“First, an abstraction isn't just a way to manage cognitive load; it's a shared surface where application, reuse, and review attach.” That is some seriously convoluted writing.
Agreed. Turned into some ai slop quickly.
Re: Retire the Abstractions
#20Following this argument to its natural conclusion, we should tell LLMs to generate the binary directly. It's still an abstraction, but it's the lowest level we can go to on current consumer hardware.
I don't think raw machine code (not even assembly) is the best form to reason about program logic. It just takes too many steps to execute sophisticated processes to keep them all in your head at once - or keep them all in an LLM context at once.