How do you guys batch requests in PHP? You don't, right? So this is an intermediate layer basically, and it sends requests every few millisecods and waits to batch things in between?
Facebook open sources Haxl
21–30 of 80 posts
Re: Facebook open sources Haxl
#22Earlier quoted context omitted.
Why Haskell?
Ah, my favorite question. We previously had a custom DSL and it outgrew it's DSL-ness. The DSL was really good at one thing (implicit concurrency and scheduling io), and bad at everything else (cpu, memory, debugging, tooling). The predecessor was wildly successful and created new problems. Once all those secondary concerns became first order, we didn't want to start building all this ecosystem stuff for our homemade…
How about Scala?
Re: Facebook open sources Haxl
#23This is a fascinating project. Haxl is the brainchild of former Glasgow Haskell Compiler lead* Simon Marlow. The tl;dr of Haxl: what if you could describe accessing a data store (a la SQL) and have the compiler and library work together to "figure out" the most efficient way to perform queries, including performing multiple queries in parallel? That's what Haxl does, it allows you to specify the "shape" of your query…
He is still committing, but not quite so often :)
Re: Facebook open sources Haxl
#24Hi. I'm one of the engineers who has worked on this so if anyone has any specific questions I can help answer and/or get someone to answer. As said by @nbm, we also have a blog post up: https://code.facebook.com/posts/302060973291128/open-sourcin... .
[deleted]
Re: Facebook open sources Haxl
#25Earlier quoted context omitted.
How large has the haskell team at Facebook gotten (unless there isn't an official group and its on a project by project bases)?
Haxl is the only team using Haskell in prod, so the team itself isn't all that large. With the traction we gain, though, we could grow.
I'm confused, Haxl is an open source library, right? So what are you using Haxl for at facebook?
Re: Facebook open sources Haxl
#26This is a fascinating project. Haxl is the brainchild of former Glasgow Haskell Compiler lead* Simon Marlow. The tl;dr of Haxl: what if you could describe accessing a data store (a la SQL) and have the compiler and library work together to "figure out" the most efficient way to perform queries, including performing multiple queries in parallel? That's what Haxl does, it allows you to specify the "shape" of your query…
He also wrote a book called Parallel and Concurrent Programming in Haskell[1], which I've heard is an excellent read[2].
[1] http://chimera.labs.oreilly.com/books/1230000000929
[2] http://www.serpentine.com/blog/2014/03/18/book-review-parall...
Re: Facebook open sources Haxl
#27Earlier quoted context omitted.
Ah, my favorite question. We previously had a custom DSL and it outgrew it's DSL-ness. The DSL was really good at one thing (implicit concurrency and scheduling io), and bad at everything else (cpu, memory, debugging, tooling). The predecessor was wildly successful and created new problems. Once all those secondary concerns became first order, we didn't want to start building all this ecosystem stuff for our homemade…
How about Scala?
Anybody that's made a serious go of FP in Scala (applicative, monad, etc) has found that it's not worth the hassle. It fights you the whole way.
Re: Facebook open sources Haxl
#28Earlier quoted context omitted.
Haxl is the only team using Haskell in prod, so the team itself isn't all that large. With the traction we gain, though, we could grow.
I'm confused, Haxl is an open source library, right? So what are you using Haxl for at facebook?
The blog post - https://code.facebook.com/posts/302060973291128/open-sourcin... - describes the initial usage - to assist the Sigma system in answering questions like "Is this content spam?".
Some background on that system, including the FXL component that Haxl replaces, is here: http://research.microsoft.com/en-us/projects/ldg/a10-stein.p...
Re: Facebook open sources Haxl
#29How do you guys batch requests in PHP? You don't, right? So this is an intermediate layer basically, and it sends requests every few millisecods and waits to batch things in between?
The systems described here isn't directly involved with any PHP code.
Like all Facebook services, they are communicated with over the Thrift RPC system, and may have PHP (or any other language) clients, and may talk to other services using Thrift (or occasionally other protocols), some of which may use PHP.
If you're asking a general question about batching requests in PHP, http://docs.hhvm.com/manual/en/hack.async.php may be informative.
Re: Facebook open sources Haxl
#30I'm an engineer on the Haxl project and am really excited to launch this today. Ask me anything!
I'm curious how this is executed.
Is it like a query engine, where you work with the entire query up-front, apply transforms and build a query plan?
Or is it more like an event loop, where you run as far as you can until the code blocks on IO, batch up and send all the pending IO requests, and run further when the tasks you're blocked on resolve?