Here's our paper about the ideas behind Haxl: http://community.haskell.org/~simonmar/papers/haxl-icfp14.pd...
Facebook open sources Haxl
31–40 of 80 posts
Re: Facebook open sources Haxl
#32Slides from Marlow from 9 months ago about Haxl and it's workings https://github.com/meiersi/HaskellerZ/blob/master/meetups/20...
Re: Facebook open sources Haxl
#33How do you guys batch requests in PHP? You don't, right? So this is an intermediate layer basically, and it sends requests every few millisecods and waits to batch things in between?
The systems described here isn't directly involved with any PHP code. Like all Facebook services, they are communicated with over the Thrift RPC system, and may have PHP (or any other language) clients, and may talk to other services using Thrift (or occasionally other protocols), some of which may use PHP. If you're asking a general question about batching requests in PHP, http://docs.hhvm.com/manual/en/hack.async.p…
Cool so Hack supports async while php doesn't.
Re: Facebook open sources Haxl
#34I'm an engineer on the Haxl project and am really excited to launch this today. Ask me anything!
I'm curious how this is executed. Is it like a query engine, where you work with the entire query up-front, apply transforms and build a query plan? Or is it more like an event loop, where you run as far as you can until the code blocks on IO, batch up and send all the pending IO requests, and run further when the tasks you're blocked on resolve?
Part of the beauty is that the actual way IO (note: in this version, IO here means 'reads from the network', almost always) is scheduled is abstracted away such that we could go with either approach w/o impacting client code.
That said, the way it currently works is more like the first. You can think of the entire haxl run (program) as an AST that is given to the execution. It expands as much of the AST as possible (anything that's not IO), and anywhere it needs IO it enqueues those requests to be scheduled. Once it's explored as much as possible, it aggressively schedules the IO (deduping, batching, and overlapping the calls). Once it all comes back, it unblocks the AST where it can, and repeats the process.
This isn't necessarily the optimal scheduling (as you point out, unblocking each part of the tree as each result comes in might be better). It was specifically designed to make it easy to play with this kind of stuff later. Since the concurrency is entirely implicit the implementation is entirely abstracted away.
Re: Facebook open sources Haxl
#35This is a fascinating project. Haxl is the brainchild of former Glasgow Haskell Compiler lead* Simon Marlow. The tl;dr of Haxl: what if you could describe accessing a data store (a la SQL) and have the compiler and library work together to "figure out" the most efficient way to perform queries, including performing multiple queries in parallel? That's what Haxl does, it allows you to specify the "shape" of your query…
He also wrote a book called Parallel and Concurrent Programming in Haskell[1], which I've heard is an excellent read[2]. [1] http://chimera.labs.oreilly.com/books/1230000000929 [2] http://www.serpentine.com/blog/2014/03/18/book-review-parall...
It's a great read indeed. If anyone is interested in more concrete applications of Haskell then a read through this should be enough to convince anyone that we can do some really amazing parallel programming on top of Haskell's RTS.
Re: Facebook open sources Haxl
#36are there similar things for iOS?
Re: Facebook open sources Haxl
#37Re: Facebook open sources Haxl
#38Earlier quoted context omitted.
Anybody that's made a serious go of FP in Scala (applicative, monad, etc) has found that it's not worth the hassle. It fights you the whole way.
Scalaz?
I hang out with the Scalazzi and authors of FP in Scala on IRC. Most of them, including the original creator, would rather write Haskell and some are doing just that.
Re: Facebook open sources Haxl
#39Earlier quoted context omitted.
How large has the haskell team at Facebook gotten (unless there isn't an official group and its on a project by project bases)?
Haxl is the only team using Haskell in prod, so the team itself isn't all that large. With the traction we gain, though, we could grow.
Is Bryan O'Sullivan and the team from his Haskell-based startup Facebook acquired in 2011 still there? I sat in on a class of his a while back and remember him ruefully laughing about having to use PHP now.
Re: Facebook open sources Haxl
#40Earlier quoted context omitted.
Haxl is the only team using Haskell in prod, so the team itself isn't all that large. With the traction we gain, though, we could grow.
Is Bryan O'Sullivan and the team from his Haskell-based startup Facebook acquired in 2011 still there? I sat in on a class of his a while back and remember him ruefully laughing about having to use PHP now.
Bryan is still here. He's actually kicking off a Haskell class within Facebook at the moment - https://twitter.com/bos31337/status/475335956556705792