41–50 of 69 posts
Earlier quoted context omitted.
My issue with SQL is lack of composability and difficulty of debugging intermediate results.
Yes, SQL is poor. What could be good is relational + array model. I have some ideas on https://tablam.org , and building not just the language but the optimizer in tandem I think will be very nice.
Earlier quoted context omitted.
Right. This is the binary tree version of the algorithm, and is nice and concise, very readable. What would take it to the next level for me is the version in the stack monoid paper, which chunks things up into workgroups. I haven't done benchmarks against the Pareas version (unfortunately it's not that easy), but I would expect the workgroup optimized version to be quite a bit faster.
I've been playing with one using scans. too bad that's not really on the map for architectural reasons, it opens up a lot of uses.
SQL. It is a joke, but an SQL engine can be massively parallel. You just don't know it, it just gives you what you want. And in many ways the operations resembles what you do for example in CUDA. CUDA backend for DuckDB or Trino would be one of my go-to projects if i was laid off.
My issue with SQL is lack of composability and difficulty of debugging intermediate results.
• Datalog is much, much better on these axes.
• Tutorial D is also better than SQL.
There were a few languages designed specifically for parallel computing spurred by DARPA's High Productivity Computing Systems project. While Fortress is dead, Chapel is still being developed.
Those languages were not effective in practice. The kind of loop parallelism that most people focus on is the least interesting and effective kind outside of niche domains. The value was low. Hardware architectures like Tera MTA were much more capable but almost no one could write effective code for them even though the language was vanilla C++ with a couple extra features. Then we learned how to write similar softwa…
SQL. It is a joke, but an SQL engine can be massively parallel. You just don't know it, it just gives you what you want. And in many ways the operations resembles what you do for example in CUDA. CUDA backend for DuckDB or Trino would be one of my go-to projects if i was laid off.
If you want to work in data engineering for massive datasets (many petabytes) pls hit me up!
ctrl-f Erlang Nothing yet? Damn...
The thing i would really like to see is some research on how to run the Erlang concurrency model on a GPU.
SQL. It is a joke, but an SQL engine can be massively parallel. You just don't know it, it just gives you what you want. And in many ways the operations resembles what you do for example in CUDA. CUDA backend for DuckDB or Trino would be one of my go-to projects if i was laid off.
Sometimes I have a problem, I just generate bunch of "possible solutions" with a constraint solver (e.g. Minizinc) which generates GBs of CSVs describing bunch of solutions, then let DuckDB analyze which ones are suitable, DuckDB is amazing.
Went in thinking "Have you heard of Go?"... but this turned out to be about GPU computing.