Has someone written the example in "native" GPU (C/Cuda) to compare performance?
Bend: a high-level language that runs on GPUs (via HVM2)
211–220 of 269 posts
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#212Very nice!
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#213For what it's worth, I ported the sum example to pure python. def sum(depth, x): if depth == 0: return x else: fst = sum(depth-1, x*2+0) # adds the fst half snd = sum(depth-1, x*2+1) # adds the snd half return fst + snd print(sum(30, 0)) under pypy3 it executes in 0m4.478s, single threaded. Under python 3.12, it executed in 1m42.148s, again single threaded. I mention that because you include benchmark information: CP…
"Thread" term in GPUs and CPUs mean different things, it's more like a SIMD lane in GPUs. A bit like ISPC can compile your code so there's eg 32 invocations of your function per CPU thread running on the same time (if you're using 16-bit datums on AVX512), and you could have 2048 executions going on after multiplying up 32 cores * 2 SMT threads/core * 32 compiler executions.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#214A lot of negativity in these threads. I say ~cudas~ kudos to the author for getting this far! The only similar project I'm aware of is Futhark, and that's haskell-y syntax - great for some people, but to the general class of C/C++/Python/Js/Java/etc. devs pretty arcane and hard to work with. My biggest complaint with this is, unlike Futhark, it only targets Cuda or multi-core. Futhark which can target OpenCL, Cuda, I…
ParaSail also goes into that direction https://github.com/parasail-lang/parasail.
Made by the designer for Ada since 1995, Tucker Taft. Some of the parallel features of ParaSail made it into Ada 2022.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#215OP comes around with some of the coolest things posted in HN recently, and all he gets is extensive criticism, when it is clear that this is an early version :/
I would be pretty appreciated if people criticize my project. That is how you grow. If people tend hide cruel truth behind applause, the world would just crumbled.
My observation is that most criticism is useless, because people don't understand why you did things the way you did them.
If you explain why, they either still don't understand, or don't agree.
If the first iPhone had been presented on HN/Reddit/Twitter, everyone would criticize the lack of physical keyboard.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#216For what it's worth, I ported the sum example to pure python. def sum(depth, x): if depth == 0: return x else: fst = sum(depth-1, x*2+0) # adds the fst half snd = sum(depth-1, x*2+1) # adds the snd half return fst + snd print(sum(30, 0)) under pypy3 it executes in 0m4.478s, single threaded. Under python 3.12, it executed in 1m42.148s, again single threaded. I mention that because you include benchmark information: CP…
Why `+0`, is this not a pointless no-op?
Yes, but when looking at the source it's more obvious this is a repeating pattern.
"Hey, I'm accessing the 0th element here, just want to make that clear"
Without the +0, that statement looks disconnected from the +1 even though conceptually its the same.
Say somebody adds some special marker/tombstone/whatever into element 0 and now all those additions need to be bumped up by one. Someone else may go and see the +1, +2, +3 and just change them to +2, +3 +4, etc while completely missing the lone variable by itself as its visually dissimilar.
Ive usually seen it used in longer lists of statements. It also keeps everything lined up formatting wise.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#217Earlier quoted context omitted.
Bend has no tail-call optimization yet. It is allocating a 1-billion long stack, while C is just looping. If you compare against a C program that does actual allocations, Bend will most likely be faster with a few threads. Bend's codegen is still abysmal, but these are all low-hanging fruits. Most of the work went into making the parallel evaluator correct (which is extremely hard!). I know that sounds "trust me", bu…
>> Bend has no tail-call optimization yet. I've never understood the fascination with tail calls and recursion among computer science folks. Just write a loop, it's what it optimises to anyway.
Computer science traditionally focuses on algorithms. Divide-and-conquer algorithms are frequently much more clearly expressed as recursion. Quicksort is the classic example, also various tree operations.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#218OP comes around with some of the coolest things posted in HN recently, and all he gets is extensive criticism, when it is clear that this is an early version :/
I would be pretty appreciated if people criticize my project. That is how you grow. If people tend hide cruel truth behind applause, the world would just crumbled.
What you appreciate has little to do with whether we should assume others are thick-skinned. If someone has always been knocked down they will struggle to positively accept criticism regardless of how well meant it might be.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#219The website claims "automatically achieves near-ideal speedup" 12x for 16x threads 51x for 16.000x threads Can someone point me to a website where it explains that this is the "ideal speedup"? Is there a formula?
It's not.
Re: Bend: a high-level language that runs on GPUs (via HVM2)
#220Wow, Bend looks like a nice language. > That's a 111x speedup by doing nothing. No thread spawning, no explicit management of locks, mutexes. We just asked bend to run our program on RTX, and it did. Simple as that. Note that, for now, Bend only supports 24-bit machine ints (u24), thus, results are always mod 2^24. Ahh, not even 32bit? Hmm, that seems pretty arbitrary for someone not accustomed to gpu's and wanting t…
> Ahh, not even 32bit?
This is a proof of concept version which focuses on the provable correctness of the parallel compiler.