VHDL?
I want a good parallel language [video]
31–40 of 69 posts
Re: I want a good parallel language [video]
#32VHDL?
I almost mentioned it in the talk, as an example of a language that's deployed very successfully and expresses parallelism at scale. Ultimately I didn't, as the core of what I'm talking about is control over dynamic allocation and scheduling, and that's not the strength of VHDL.
Re: I want a good parallel language [video]
#33prolog?
Now you are backtracking (pun intended).
Re: I want a good parallel language [video]
#34Earlier quoted context omitted.
is it a language problem though? it's just lack of tooling.
The dataframe paradigm (a good example being polars) is another good alternative that's more composable (imo).
It is true. I still hate it. I think because it always offers 10 different ways to do the same thing. So it is just too much to remember.
Re: I want a good parallel language [video]
#35There were a few languages designed specifically for parallel computing spurred by DARPA's High Productivity Computing Systems project. While Fortress is dead, Chapel is still being developed.
Those languages were not effective in practice. The kind of loop parallelism that most people focus on is the least interesting and effective kind outside of niche domains. The value was low.
Hardware architectures like Tera MTA were much more capable but almost no one could write effective code for them even though the language was vanilla C++ with a couple extra features. Then we learned how to write similar software architecture on standard CPUs. The same problem of people being bad at it remained.
The common thread in all of this is people. Humans as a group are terrible at reasoning about non-trivial parallelism. The tools almost don't matter. Reasoning effectively about parallelism involves manipulating a space that is quite evidently beyond most human cognitive abilities to reason about.
Parallelism was never about the language. Most people can't build the necessary mental model in any language.
Re: I want a good parallel language [video]
#36SQL. It is a joke, but an SQL engine can be massively parallel. You just don't know it, it just gives you what you want. And in many ways the operations resembles what you do for example in CUDA. CUDA backend for DuckDB or Trino would be one of my go-to projects if i was laid off.
My issue with SQL is lack of composability and difficulty of debugging intermediate results.
Yes, SQL is poor.
What could be good is relational + array model. I have some ideas on https://tablam.org, and building not just the language but the optimizer in tandem I think will be very nice.
Re: I want a good parallel language [video]
#37It seems like there are two sides to this problem, both of which are hard and go hand in hand. There is the HCI problem of having abstractions are rich enough to handle problems like parsing and scheduling on the GPU. Then you need a sufficiently smart compiler problem of lowering these problems to the GPU. But of course, there's a limit to how smart a compiler can be, which loops back to your abstraction design.
Overall, it seems to be a really interesting problem!
Re: I want a good parallel language [video]
#38Earlier quoted context omitted.
(author here) check_brackets_bt is actually exactly the algorithm that Raph mentions
Right. This is the binary tree version of the algorithm, and is nice and concise, very readable. What would take it to the next level for me is the version in the stack monoid paper, which chunks things up into workgroups. I haven't done benchmarks against the Pareas version (unfortunately it's not that easy), but I would expect the workgroup optimized version to be quite a bit faster.
To be clear, you can express workgroup parallelism in Futhark, or rather, if the compiler sees that you've programmed your problem in such a way that it can take advantage of workgroup parallelism, it will.
But you're right, it would be interesting to see how the different approaches stack up to each other. The Pareas project linked above also includes an implementation using radix sort.
Re: I want a good parallel language [video]
#39Re: I want a good parallel language [video]
#40ctrl-f Erlang
Nothing yet? Damn...