Beating C with Futhark Running on GPU • Hacker News Board

Re: Beating C with Futhark Running on GPU

#71

Post by hellofunk » Fri, Oct 25, 2019, 8:25 PM UTC

Athas wrote: post #70

hellofunk wrote: post #69

A lot of the Futhark demos you see are rather basic algorithms like matrix multiplication, and the documentation for Futhark does say that it is not well-suited to complex kernels, so that to me puts a big limiting factor on how useful it could be to invest in it. I really like technologies like this and Sycl which aim to greatly simplify the process of writing GPU code. The important thing is that it can handle what…

It depends on what you consider a "complex kernel". Futhark is only for regular non-recursive data parallelism, but I'll argue that something like a genetic algorithm that does calibration of market parameters in the Heston model[0] is pretty complex. It comprises multiple levels of parallelism and several kernels (last I checked, the core work is done in four kernels which are invoked in a loop). But more importably…

Thank you for this thoughtful reply, really appreciate it.

Re: Beating C with Futhark Running on GPU

#72

Post by rrss » Fri, Oct 25, 2019, 9:09 PM UTC

Mathnerd314 wrote: post #59

e12e wrote: post #47

> Word counting is primarily IO-bound, and it is much too expensive to ferry the file contents all the way to the GPU over the (relatively) slow PCI Express bus just to do a relatively meagre amount of computation. After seeing that it's possible to play crysis using software rendering on an AMD Rome cpu with 128 hw threads [1] - might this lead to some vindication for AMD sticking with opencl (assuming exposing such…

CUDA is proprietary to NVidia, and is pretty much the standard for GPU computing. AMD's been chipping away with OpenCL, Vulkan/GLSL, https://github.com/RadeonOpenCompute/hcc/wiki , etc. but not much luck so far. I wouldn't say AMD's been "sticking with" OpenCL, if anything it seems like they will deprecate it in a few years, as the plan is to fold OpenCL into Vulkan. I guess it is possible to use OpenCL on the CPU as…

Last I checked, AMD implemented the cuda apis as "hip."

Re: Beating C with Futhark Running on GPU

#73

Post by pjmlp » Fri, Oct 25, 2019, 9:12 PM UTC

flohofwoe wrote: post #29

pjmlp wrote: post #25

Earlier quoted context omitted.

Sure it was, there were zero reasons to use C on CP/M, MS-DOS, Atari, Amiga, Mac. It was just another programming language fighting for developer eyes. On Windows and OS/2, although IBM and Microsoft decided to go with C for the underlying low level layers, C++ was the way to go for high level coding, C Set++, MFC. With Borland having Turbo Vision, OWL and VCL. Macs were Object Pascal territory, and when MPW got C an…

IME C++ wasn't really a mainstream option until the second half of the 90's. BeOS choosing C++ for its operating system APIs was an extremely exotic choice at the time (same level of "exotic" as NeXT choosing Objective-C). And IMHO, at that time, before or around C++98, C++ didn't fix a single problem of C, but instead just added a lot of new ones (one could argue that this is still the case even today).

It was so exotic that at my university, FCT/UNL, starting in 1992 they switched the first year students to learn Pascal followed by C++.

C was never taught as such, as any student was expected to know it from their C++ classes.

The professor was a great teacher of what all the ways that C++ fixed C's problems, by providing his own data structures for strings, arrays, vectors, linked lists and hash tables, all with bounds checking enabled by default on his implementation.

Other issues that C++ fixed over C was having implicit conversions, a proper way to allocate memory (malloc() with sizeof, really?), ability to ensure valid pointers via references.

Re: Beating C with Futhark Running on GPU

#74

Post by pjmlp » Fri, Oct 25, 2019, 9:14 PM UTC

flohofwoe wrote: post #28

pjmlp wrote: post #17

Earlier quoted context omitted.

The same reason people got to choose JavaScript or PHP years later, the platform's adoption, in this case UNIX.

C was (and still is) a fairly obvious choice for programming when you need full control over the memory layout of an application (which is becoming all the more important with the growing CPU/memory gap). I choose C 15 years before I got into contact with UNIX (first on the Amiga, after that on Windows, and only fairly recently macOS and Linux).

Nothing special about C, plenty of other languages offer similar features.

ISO C doesn't offer full control over anything beyond the abstract memory model of the standard.

Re: Beating C with Futhark Running on GPU

#75

Post by pjmlp » Fri, Oct 25, 2019, 9:15 PM UTC

namirez wrote: post #37

pjmlp wrote: post #17

Earlier quoted context omitted.

The same reason people got to choose JavaScript or PHP years later, the platform's adoption, in this case UNIX.

C is not a fad; it's an outlier among languages in the sense that basically it's a portable assembler very close to the metal. If you change the basic architecture of the machine, you can create a better language. For now however it's unlikely to beat C.

Portable assembler for an abstract machine modelled on what PDP-11 processors used to be.

Re: Beating C with Futhark Running on GPU

#76

Post by pjmlp » Fri, Oct 25, 2019, 9:16 PM UTC

megous wrote: post #46

gmueckl wrote: post #41

Earlier quoted context omitted.

I'm not sure that this high level assembler assumption still holds for SIMD-capable CPUs. C compilers are asked to do quite drastic code transformations like autovectorization on these architectures. With these, the tight relationship between the high level code C code and the generated machine code is removed.

You can still treat it that way even with SIMD. I quite enjoy using NEON (ARM SIMD) intrinsics in C, and the like.

Any language can have intrinsics, in fact the first systems language with intrinsics support appeared 10 years before C was created.

Re: Beating C with Futhark Running on GPU

#77

Post by jstimpfle » Fri, Oct 25, 2019, 10:05 PM UTC

pjmlp wrote: post #73

flohofwoe wrote: post #29

Earlier quoted context omitted.

IME C++ wasn't really a mainstream option until the second half of the 90's. BeOS choosing C++ for its operating system APIs was an extremely exotic choice at the time (same level of "exotic" as NeXT choosing Objective-C). And IMHO, at that time, before or around C++98, C++ didn't fix a single problem of C, but instead just added a lot of new ones (one could argue that this is still the case even today).

It was so exotic that at my university, FCT/UNL, starting in 1992 they switched the first year students to learn Pascal followed by C++. C was never taught as such, as any student was expected to know it from their C++ classes. The professor was a great teacher of what all the ways that C++ fixed C's problems, by providing his own data structures for strings, arrays, vectors, linked lists and hash tables, all with bo…

C++ fixes many of C's problems and introduces so many more that are not fixable. ("But why don't you write const-correct code? Why don't you use move semantics? Use the rule of three!")

> malloc() with sizeof, really?

One tiny macro to solve 20% of all the problems people are whining about.

    void _alloc_memory(void **ptr, size_t numElems, size_t elemSize)
    {
        size_t numBytes = safe_multiply(numElems, elemSize);
        void *p = malloc(numBytes);
        if (p == NULL)
            fatal("OOM!\n");
        *ptr = p;
    }

    #define ALLOC_MEMORY(ptr, numElems) _alloc_memory((ptr), (numElems), sizeof **(ptr))

    int *myArray;
    ALLOC_MEMORY(&myArray, 25);

I've been using this for years without a problem.

Re: Beating C with Futhark Running on GPU

#78

Post by jstimpfle » Fri, Oct 25, 2019, 10:16 PM UTC

pjmlp wrote: post #74

flohofwoe wrote: post #28

Earlier quoted context omitted.

C was (and still is) a fairly obvious choice for programming when you need full control over the memory layout of an application (which is becoming all the more important with the growing CPU/memory gap). I choose C 15 years before I got into contact with UNIX (first on the Amiga, after that on Windows, and only fairly recently macOS and Linux).

Nothing special about C, plenty of other languages offer similar features. ISO C doesn't offer full control over anything beyond the abstract memory model of the standard.

So what? Chill out.

We all know that C is not without flaws. I would use other languages if there were good alternatives. For example, I liked some parts of Delphi, but it has too many show stoppers. For example, all local variables still have to be declared in the variables section before the function body, right? And are we still required to make type aliases to use pointer types in important places, such as function signatures?

C is special in that it is the only language that I've known that has a minimalistic attitude, resulting in a shitty language (every language is shit!) whose problems we can actually work around in practice.

Re: Beating C with Futhark Running on GPU

#79

Post by ngcc_hk » Fri, Oct 25, 2019, 10:48 PM UTC

No time working on my I Ching course for my master degree. However I join a programming competition many years ago About an incident of a crazy assignment for a kid. My clone of the github here:

https://github.com/kwccoin/ABCDEFGHPPP

It is fun. I think I even try to use a micro version of cobol to do it. But the fastest is still c.

May be we can start one. Sadly no time to join.

(I think there is a web site that post many versions of the same program. It must have Wc. If not these should be there. )

Re: Beating C with Futhark Running on GPU

#80

Post by ummaycoc » Sat, Oct 26, 2019, 1:10 AM UTC

Athas wrote: post #13

yiyus wrote: post #2

Very honest discussion of the results. I liked it. Would it be possible to use Futhark to rewrite the APL implementation instead of the Haskell one? That would make an interesting comparison.

> Would it be possible to use Futhark to rewrite the APL implementation instead of the Haskell one? That would make an interesting comparison. Sadly, from what I can see, the APL version makes use of so-called nested arrays in the 'words' function, specifically arrays of strings (this is different from multidimensional arrays). Futhark does not directly support nested arrays. A rewrite of the APL implementation would…

In my original version I didn't use a nested array of strings, but [Olzd](https://news.ycombinator.com/user?id=olzd) pointed out a way you could do that and I added it as a theoretical first attempt (theoretical since it wasn't my first attempt, but would have been had I been better at APL).

My first attempt did some stuff with subtracting items in an array from their neighbor. Now with info I got from [mlochbaum](https://news.ycombinator.com/user?id=mlochbaum) I have another version that uses windowed reductions.

So those are three versions there; after that I just split it up just to see where that leads and that actually ends up feeling a lot like the Haskell / Futhark solution to me.