21–30 of 34 posts
I have not fully digested yet, but seems very similar to Scala Parallel Collections and Java8 Streams. There are databases which implements such interfaces.
I have not fully digested yet, but seems very similar to Scala Parallel Collections and Java8 Streams. There are databases which implements such interfaces.
It isn't the slightest bit similar to those! Haxl is a high-level library for specifying (and optimising) concurrent data retrieval.
I have not fully digested yet, but seems very similar to Scala Parallel Collections and Java8 Streams. There are databases which implements such interfaces.
Which databases are those and what are the relevant features? Are you talking about FILESTREAM? https://technet.microsoft.com/en-us/library/bb933993(v=sql.1...
Earlier quoted context omitted.
It isn't the slightest bit similar to those! Haxl is a high-level library for specifying (and optimising) concurrent data retrieval.
I'm not sure that's an apt description either. Haxl's features include auto batching of IO through parallelism and a temporal cache...for error logging when failures occur. It's spelled out in the first minute he talks about it. In the IO batching code (the boilerplate) you specify how and what to output in your logging. Haxl can be used without performing optimization (write an inefficient batcher) or doing data ret…
EDIT: the Haxl docs describe it as "a library and EDSL for efficient scheduling of concurrent data accesses with a concise applicative API"
Earlier quoted context omitted.
Even in Haskell you still have to write some boilerplate to get this by default. Languages that don't separate pure computation from IO (and other side-effects) make it even harder on the compiler. So to answer your question: implementations will do this by default only after pure/constant will become the default for all functions/variables, with side-effecting/mutable clearly marked.
If one looks at it from a 'what a today's compiler can do', then sure, one needs to statically declare everything. If one looks at things from a 'any technical task can be handled by machine learning, sooner or later' point of view, there seems to be no obvious reason why parallelization, which is a purely technical task, not like say, writing music, could not sooner or later be handled by machine learning algorithms…
The idea here is that we want to find a way to make automatic parallelization of computer programmes happen as much as possible long before we've solved general AI.
So even in a language like Haskell you still have to pay some attention to make your code amenable to parallelism. See Guy Steele's advice (see https://vimeo.com/6624203).
Earlier quoted context omitted.
Analysis of real software that has looked for the maximum theoretical implicit concurrency has found that software to have its implicit concurrency level top out very quickly, often not even making it up to 2x. You don't hear about this today not because nobody has tried it, but because it's been tried and it hasn't been found worth the bother. Amdahl's law is a real jerk sometimes. If you want concurrent code, you'r…
Sadly it seems like the youngsters are going in the opposite direction these days. It's the old fogies who have to keep saying, I've got a functor, I've got a monoid, I've got a monoid functor = monad.
I wonder how long it will be before compilers/interpreters of async-aware languages just do this by default. CPUs and low-level language compilers jump through all kinds of hoops of out-of-order execution, branch prediction, caching, parallel execution, etc. I picture a day maybe 10 years from now where developers in most languages don't even have to think about these things. All the old-timers will still be structur…
Earlier quoted context omitted.
Which databases are those and what are the relevant features? Are you talking about FILESTREAM? https://technet.microsoft.com/en-us/library/bb933993(v=sql.1...
Hazelcast Jet, Spark, my project... they work similar way. But as I wrote, I need to read the presentation first.
Earlier quoted context omitted.
Sadly it seems like the youngsters are going in the opposite direction these days. It's the old fogies who have to keep saying, I've got a functor, I've got a monoid, I've got a monoid functor = monad.
Aren't Applicative Functors still all the rage? Monads are not all that well suited to automatic parallelization (as Simon Marlow points out, too).
How does this differ from BEAM langs which already make concurrency "unreasonably easy"?
For one, they serve different purposes. Haxl is specifically for concurrent data retrieval while BEAM is a general purpose platform for fault-tolerant computation. For another, they operate via different interfaces. BEAM languages communicate concurrently only via a message passing interface. Haxl lets the author write declarative code specifying what to retrieve, then the library executes it concurrently and in para…