Haxl: Making Concurrency Unreasonably Easy [video]
events.techcast.com
Haxl: Making Concurrency Unreasonably Easy [video]
1–10 of 34 posts
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#2Great talk.
For more on concurrency and parallelism in Haskell, check out Parallel and Concurrent Programming in Haskell [0], deemed as the best book on the subject, also written by Simon Marlow.
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#3I have not fully digested yet, but seems very similar to Scala Parallel Collections and Java8 Streams. There are databases which implements such interfaces.
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#4I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through all kinds of hoops of out-of-order execution, branch prediction, caching, parallel execution, etc.
I picture a day maybe 10 years from now where developers in most languages don't even have to think about these things. All the old-timers will still be structuring their code "as though it didn't exist" whereas the new kids will fly along without even thinking about it. Kind of like garbage collection the first few years.
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#5[deleted]
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#6I looked through the slides but not the video and the slides ignore the hard problem: how do you schedule these requests? How do you know how many parallel requests you can issue without hammering the database or service? How do you batch queries so that you get acceptable latency and a query size that will not choke the database?
The last question is probably easy for most use cases where you have independent requests coming in (typical web application) - in the context of a single request you can usually get away with batching as much as is possible. But the scheduling problem is very similar to the promises of "free parallelism because Church-Rosser" - actually taking advantage is an open problem. Even when you know how much time each job takes in advance, multiprocessor scheduling is NP-hard.
Anyway, if someone watched the video and the question is addressed there, please let me know so I can watch it.
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#7I looked through the slides but not the video and the slides ignore the hard problem: how do you schedule these requests? How do you know how many parallel requests you can issue without hammering the database or service? How do you batch queries so that you get acceptable latency and a query size that will not choke the database? The last question is probably easy for most use cases where you have independent reques…
I think the idea is that with the Haxl approach the scheduling can be dealt with independently from your business logic.
By the way, I don't see how Church-Rosser would give you any free parallelism---even in theory. You'd still have to heed Guy Steele's advice (see https://vimeo.com/6624203).
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#8I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through all kinds of hoops of out-of-order execution, branch prediction, caching, parallel execution, etc. I picture a day maybe 10 years from now where developers in most languages don't even have to think about these things. All the old-timers will still be structur…
Even in Haskell you still have to write some boilerplate to get this by default. Languages that don't separate pure computation from IO (and other side-effects) make it even harder on the compiler.
So to answer your question: implementations will do this by default only after pure/constant will become the default for all functions/variables, with side-effecting/mutable clearly marked.
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#9I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through all kinds of hoops of out-of-order execution, branch prediction, caching, parallel execution, etc. I picture a day maybe 10 years from now where developers in most languages don't even have to think about these things. All the old-timers will still be structur…
Analysis of real software that has looked for the maximum theoretical implicit concurrency has found that software to have its implicit concurrency level top out very quickly, often not even making it up to 2x. You don't hear about this today not because nobody has tried it, but because it's been tried and it hasn't been found worth the bother. Amdahl's law is a real jerk sometimes.
If you want concurrent code, you're going to have to write it with an awareness of concurrency issues. There are things that languages can do to make it easier, but in terms of doing it automatically even that hasn't had a great track record. Fortress [1] took a good run at it, but ultimately didn't seem to get that far, at least relative to their ambitions.
Personally I suspect that while Haskell may not be that language today, and may not be that language ever necessarily, that the "magically concurrent" language of the future is going to end up looking a lot like Haskell, in particular, with the ability and need to consider the mathematical properties of your code for things like associativity and commutativity. It's going to take a pretty major sea change in a whole lot of attitudes before "the youngsters" are just casually reeling off group theory names while the "old fogies" are stuck going "What's a monoid again?" Not impossible, but hard to imagine how we get to there from here.
[1]: https://en.wikipedia.org/wiki/Fortress_(programming_language...
Re: Haxl: Making Concurrency Unreasonably Easy [video]
#10How does this differ from BEAM langs which already make concurrency "unreasonably easy"?