Live data from Hacker News

Bend: a high-level language that runs on GPUs (via HVM2)

github.com

261–269 of 269 posts

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#261

Earlier quoted context omitted.

Bend has no tail-call optimization yet. It is allocating a 1-billion long stack, while C is just looping. If you compare against a C program that does actual allocations, Bend will most likely be faster with a few threads. Bend's codegen is still abysmal, but these are all low-hanging fruits. Most of the work went into making the parallel evaluator correct (which is extremely hard!). I know that sounds "trust me", bu…

>> Bend has no tail-call optimization yet. I've never understood the fascination with tail calls and recursion among computer science folks. Just write a loop, it's what it optimises to anyway.

I've never understood the fascination with programming languages among computer science folks. Just write machine code directly, it's what it compiles to anyway.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#263

The website claims "automatically achieves near-ideal speedup" 12x for 16x threads 51x for 16.000x threads Can someone point me to a website where it explains that this is the "ideal speedup"? Is there a formula?

Bend is intriguing -- 1. Some potentially useful perspectives: * Weak scaling vs strong scaling: https://www.kth.se/blogs/pdc/2018/11/scalability-strong-and-... ? * ... Strong scaling, especially comparing to a modern sequential baseline, seems to be where folks are noting the author still has some work to do wrt getting to ideal speedups for what performance people care about * There are parallel models of computati…

Thanks for the long reply.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#264

Earlier quoted context omitted.

I agree... Just a note: we are NOT 10x slower than Python. I think a lot of people got the wrong message from this thread. HVM is actually quite fast already. It is just that, on this specific program, Python was doing no allocations, while HVM was allocating a lot. If you compare programs that do the same allocation, HVM already outperforms not just Python but even compiled languages like Haskell/GHC, due to using a…

If you wanna be really honest you write one short example where HVM is weak and one where it's strong, use the opportunity to explain why if you want

That is actually an amazing idea. I'll adopt it.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#265

Earlier quoted context omitted.

It is not in alpha, nor not ready. You can use it in production today, if you want to. It is just not fast . That is different. CPython is still 100x slower than C, and is widely deployed in practice.

Seems like these are major problems for software whose whole purpose appears to make parallelizable programs go faster... Maybe I just don't understand the point then. To me it appears like a cool tech demo that fails to achieve the actual goal of delivering performance increases (by better utilizing the hardware), but it sounds like from your reply that being a cool tech demo that is probably not actually practical…

While it is not fast in a single-thread, it is still 5x-7x faster than Node.js today for programs that are allocate a lot. If all you want is to run a program faster, and doesn't mind a bit more energy, Bend could be useful for you today.

And that's comparing a first-version interpreter against a SOTA runtime deployed in all browsers around the world and optimized by all major companies over 20+ years. If that's not useful to you, that's useful to me, which is why I wanted to share so it can be useful to more people.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#266

Earlier quoted context omitted.

> it is allocating 2 IC nodes for each numeric operation, while Python is not While that's true, Python would be using big integers (PyLongObject) for most of the computations, meaning every number gets allocated on the heap. If we use a Python implementation that would avoid this, like PyPy or Cython, the results change significantly: % cat sum.py def sum(depth, x): if depth == 0: return x else: fst = sum(depth-1, x…

The only claim I made is that it scales linearly with cores. Nothing else! I'm personally putting a LOT of effort to make our claims as accurate and truthful as possible, in every single place. Documentation, website, demos. I spent hours in meetings to make sure everything is correct. Yet, sometimes it feels that no matter how much effort I put, people will just find ways to misinterpret it. We published the real be…

> I'm personally putting a LOT of effort to make our claims as accurate and truthful as possible, in every single place.

I'm not informed enough to comment on the performance but I really like this attitude of not overselling your product but still claiming that you reached a milestone. That's a fine balance to strike and some people will misunderstand because we just do not assume that much nuance – and especially not truth – from marketing statements.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#267

I remember seeing HVM on here a year or two back when it came out and it looked intriguing. Exciting to see something being built on top of it! I would say that the play on words that gives the language its name ("Bend") doesn't really make sense... https://github.com/HigherOrderCO/bend/blob/main/GUIDE.md > Bending is the opposite of folding. Whatever fold consumes, bend creates. But in everyday language bending is n…

Re: name. Fold and bend are indeed called fold and unfold in Haskell and traditional functional programming literature.

I wonder if bend has to do with how we manipulate the computation's interaction graph while evaluating a bend. There might be some bending of wires!

Re: code example

In the code example, x=0 is the seed value. tree = fork(0) must mean "fork off to evaluate the bend at the seed value". In that first fork, we fork twice with the value x=1, to get the left and right subtrees of the top level node. We then fork four instances of x=2, eight instances of x = 3, and finally get our balanced binary tree with eight 7s.

Note this is guesswork. I don't know what the ![a, b] syntax means, and I haven't read much of the guide.

Appendix: Notes on Fold Vs Bend

I wrote these for an earlier draft while reminding myself about these operations. I include them more for my benefit, and in case they help you or the audience.

Fold and bend are categorical duals, aka catamorphisms and anamorphisms. One takes a monadic value and reduces it into an ordinary value. The other takes an ordinary value and expands it into a comonadic value.

Fold starts with a value in an inductive data type, and then replaces its constructors with a function. For example it takes a list (1:2):3, and replaces the constructor : with the function `+`, to get (1+2)+3 = 6

Bend starts with a seed value and a function taking values into constructor expressions for a conductive data type. It then grows the seed into a potentially infinite AST. For example the seed value 1 and the function f(x:xs) = (x+1) : (x:xs) gives us the infinite lazy list [1, 2, 3, ...]

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#268

I remember seeing HVM on here a year or two back when it came out and it looked intriguing. Exciting to see something being built on top of it! I would say that the play on words that gives the language its name ("Bend") doesn't really make sense... https://github.com/HigherOrderCO/bend/blob/main/GUIDE.md > Bending is the opposite of folding. Whatever fold consumes, bend creates. But in everyday language bending is n…

Re: name. Fold and bend are indeed called fold and unfold in Haskell and traditional functional programming literature. I wonder if bend has to do with how we manipulate the computation's interaction graph while evaluating a bend. There might be some bending of wires! Re: code example In the code example, x=0 is the seed value. tree = fork(0) must mean "fork off to evaluate the bend at the seed value". In that first…

The question that comes to me is: can I use fork(x) outside of a bend?

Seems like probably not, there doesn't seem to be enough information in the 'argument' to this 'function' to do anything useful without the implicit context of the bend construct.

For that reason I think I'd prefer it if fork was a keyword (like 'bend' and 'when') rather than a 'function', just at the surface syntax level to give a clue it is something special.

I guess fork is a kind of 'magic' function that represents the body of the bend. It's a bit like a 'self' or 'this'.

At the moment this syntax is in a weird half-way point ...the underlying concept is necessarily functional but it's trying to look kind of like an imperative for-loop still.

I wonder if we couldn't just explicitly create a 'bendable' recursive function that can be 'bent' by calling it. But I guess it's like this because it needs to be tightly constrained by the 'when' and 'else' forms.

TBH the more I look at this example the more confusing it is. The other part I wonder about is the assigning of new values to tree var... can I set other local vars from outside the bend scope? I don't think so, I guess it'd be a syntax error if the var names assigned in the 'when' and 'else' clauses didn't match?

Again it's sort of overloading an imperative-looking syntax to implicitly do the 'return' from the implicit recursive function.

Later on there is this example:

    def render(depth, shader):
      bend d = 0, i = 0:
        when d 
And here I wonder - does 'width' have a value after the bend? Or it's only the last assignment in each clause that is privileged?

That's an odd mix in a language which otherwise has explicit returns like Python.

If so I wonder if a syntax something like this might be clearer:

    def render(depth, shader):
      bend color with d = 0, i = 0:
        when d 
i.e. name the return var once in the bend itself, yield intermediate values (to itself, recursively) and return the final state.

Re: Bend: a high-level language that runs on GPUs (via HVM2)

#269

I remember seeing HVM on here a year or two back when it came out and it looked intriguing. Exciting to see something being built on top of it! I would say that the play on words that gives the language its name ("Bend") doesn't really make sense... https://github.com/HigherOrderCO/bend/blob/main/GUIDE.md > Bending is the opposite of folding. Whatever fold consumes, bend creates. But in everyday language bending is n…

`bend` is a convenience syntax that "just creates an in-place recursive function, immediately calls it with an initial state, and then assigns the end result to a local variable" ... "in a single statement, rather than needing to name a separate external auxilliary function that you'll only use once", according to the original author on Twitter: https://x.com/VictorTaelin/status/1791964640533958924 and https://x.com/VictorTaelin/status/1791996185449791932

But contrary to this, I think explicitly separating function declaration and function calling, in the following kind of syntax, would make it much clearer and less complected where the initial condition `tree = fork(0)` comes from. In the original example it came from `bend x = 0`, but here the function declaration is separate and the call more explicit: so it more obviously comes from `createTree(0)`:

    type Tree
      Branch { left, right }
      Leaf { value }

    def main():

      createTree(x):
        x 
Besides not needing a local variable `tree` here, the unique thing here is the elimination of the else-clause, to reduce unnecessary nesting, and a rule that the language just early returns the last result of any nested condition. If it doesn't go into any nested condition, then it just returns the last result in the main function body (like Ruby). Without any `return` keywords needed in either case. Wouldn't this be quite beautiful?
Post reply on HN