Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Retire the Abstractions

hazyresearch.stanford.edu

Post reply on HN
Print view View original

41–50 of 60 posts

Re: Retire the Abstractions

#41

Post by winter_blue »

bloaf wrote: post #29

I feel like this is the exact opposite of the conclusion I've been coming to. In an age where anyone can vibe code stuff at the drop of a hat, I want the ability to assert guarantees/contracts at a high level, and then let AI work out the details. I want to force AI to work within the confines of an abstraction, not independently of it.

> I want to force AI to work within the confines of an abstraction, not independently of it.

Yes, this is the way. We need programming languages that make this easier.

Re: Retire the Abstractions

#42

Post by augment_me »

This holds half-true.

What we decided to do is essentially just hand-rolling PTX with the agents under a harness. If there is lacking some function, we write/take a CuteDSL kernel, export PTX on compilation, transform it with the harness and then feed it into the agents.

We beat cuBLAS, and we beat all of abstractions - Triton, Helion, TLX, Mojo, ThunderKittens, cuteDSL, etc on Hopper and Blackwell for all of our workloads over 3-4 weeks.

vLLM dropped torch.compile support because they realized that they programmers were good enough to just generate the Triton kernels directly for all the passes efficiently.

If you work with this for prod the writing is on the wall sadly. The abstraction layer is just really much lower if you want full perf.

Re: Retire the Abstractions

#43

Post by rryan »

jrflo wrote: post #14
rryan wrote: post #10

Earlier quoted context omitted.

I know nothing about the PhD candidate and professor who co-authored this, but I don't think this statement makes sense in general. Typically CS labs at universities contain people who have not spent a significant amount of time exposed to large industry codebases and the corresponding complexity. I think the post would have more credibility coming from e.g. the platforms team at a tech company with a monorepo.

There is something to be argued about industry vs academic experience but this post has nothing to do with large industry codebases

Disagree. The abstractions they are talking about aren't like, the y-combinator. They're talking about boring software engineering abstractions.

Re: Retire the Abstractions

#44

Post by rryan »

JMiao wrote: post #17
rryan wrote: post #10

Earlier quoted context omitted.

I know nothing about the PhD candidate and professor who co-authored this, but I don't think this statement makes sense in general. Typically CS labs at universities contain people who have not spent a significant amount of time exposed to large industry codebases and the corresponding complexity. I think the post would have more credibility coming from e.g. the platforms team at a tech company with a monorepo.

sounds like it might help to know something about the phd candidate and professor

Heh, not really

Re: Retire the Abstractions

#45

Post by vatsachak »

augment_me wrote: post #42

This holds half-true. What we decided to do is essentially just hand-rolling PTX with the agents under a harness. If there is lacking some function, we write/take a CuteDSL kernel, export PTX on compilation, transform it with the harness and then feed it into the agents. We beat cuBLAS, and we beat all of abstractions - Triton, Helion, TLX, Mojo, ThunderKittens, cuteDSL, etc on Hopper and Blackwell for all of our wor…

But you end up writing your own abstractions in the process?

Re: Retire the Abstractions

#46

Post by vatsachak »

noncoml wrote: post #26

I kind of angree with the anrgument that for an ideal AGI abstractions are not needed. It would write everything perfectly without even needing a programming language. It would know everything to the byte and would create perfect machine code meeting all the requirements as perfect as it’s possible. But the reality is different. First of all LLM were trained on current data, with the abstractions we currently use. Se…

Even with an ideal AGI you would need abstractions because in order to talk in specifics you would need an abstraction.

Re: Retire the Abstractions

#47

Post by augment_me »

vatsachak wrote: post #45
augment_me wrote: post #42

This holds half-true. What we decided to do is essentially just hand-rolling PTX with the agents under a harness. If there is lacking some function, we write/take a CuteDSL kernel, export PTX on compilation, transform it with the harness and then feed it into the agents. We beat cuBLAS, and we beat all of abstractions - Triton, Helion, TLX, Mojo, ThunderKittens, cuteDSL, etc on Hopper and Blackwell for all of our wor…

But you end up writing your own abstractions in the process?

Ish, today I am on the fence because sometimes(half-half) its necessary, a year ago it was completely impossible, but I think its a question of context length, attention and task. Modern models can handle raw PTX pretty well. If you have below 20k lines, it's able to make its own abstractions in the thinking loop Nth pass when it needs to. The only reason we would have to make our own abstractions today would be to fill the limits of the current context context window attention trade-offs the companies make.

I think the article is calling for the redundancy of high-level tiling abstractions. There is simply no need to work at a higher level and give up performance today when code is free.

Re: Retire the Abstractions

#48

Post by vatsachak »

augment_me wrote: post #47
vatsachak wrote: post #45

Earlier quoted context omitted.

But you end up writing your own abstractions in the process?

Ish, today I am on the fence because sometimes(half-half) its necessary, a year ago it was completely impossible, but I think its a question of context length, attention and task. Modern models can handle raw PTX pretty well. If you have below 20k lines, it's able to make its own abstractions in the thinking loop Nth pass when it needs to. The only reason we would have to make our own abstractions today would be to f…

I feel like both abstractions and LLMs are time savers, so why not combine both.

Also, if I don't create abstractions and guide the LLM I've found that the codebase starts getting out of my reach. And until LLMs can maintain code ad infinitum I like to understand my code.

I can see why it might not matter in hardcore performance coding though.

Re: Retire the Abstractions

#49

Post by rhdunn »

vatsachak wrote: post #32
FjordWarden wrote: post #24

Earlier quoted context omitted.

That is not what the text is arguing for and it has many good ideas, dropping the abstractions but keeping the test harness is one of them. I've worked on many aspect of programming language implementation through the years and I would struggle with the abstractions of parser-generators and bytecode designed by committee like an enterprise Java developer struggles with the Hibernate ORM. Dunno what people are putting…

Abstractions aren't a burden though, they are more like a support. A good abstraction let's you write 10 programs in the time it would have taken to write one. Also, we now have "production" languages with generics and monomorphization so abstraction is easier than ever. Djikstra has a great article as to why English language programming will always be second to a programming language. A "formal system" (edited from…

LLMs have been trained on a lot of data and have built representations of various concepts and links between them from that training data. This is why they are good when seeing names of things like `Semigroup` as they can draw on all the information about that concept from multiple sources (mathematics papers and discussions, programming articles and code, etc.).

There was an article or comment somewhere that said that when an LLM was prompted in comments/variable names about a complex algorithm or something like that in a very specific domain it was very knowledgeable. However, when given different names/terminology for the same thing it didn't understand the problem it was prompted to solve.

This is also why LLMs are good at generating code using Bootstrap, Tailwind or any other technologies as they can relate terms like `flex` and `grid` to various layout patterns. So using sensible names from the literature/specifications/etc. helps both developers and LLMs.

Post reply on HN