Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Facebook open sources Haxl

code.facebook.com

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Facebook open sources Haxl

#51

Post by lbrandy »

hythloday wrote: post #49
lbrandy wrote: post #16

Earlier quoted context omitted.

Ah, my favorite question. We previously had a custom DSL and it outgrew it's DSL-ness. The DSL was really good at one thing (implicit concurrency and scheduling io), and bad at everything else (cpu, memory, debugging, tooling). The predecessor was wildly successful and created new problems. Once all those secondary concerns became first order, we didn't want to start building all this ecosystem stuff for our homemade…

I only skimmed the tutorial, but in Scala/Finagle: val (friendsX, friendsY) = (friendsOf(x), friendsOf(y)) for { fx really liking the look of this, thanks for open-sourcing!

As I understand this, this is an easy but still explicit concurrency construction, which is not what haxl does. It's easy to imagine how something like the toy example of friend-list-intersection could be converted to explicit concurrency in languages with good concurrency support. This is an easy optimization to make by hand in the cases where concurrency is obvious at some layer or within some abstraction. The power of haxl comes when the two (or whatever) requests come from wildly different places in the AST. For example, if you combine the result of friend-list-intersection with some other numeric quantity that's the result of some other fetches. Haxl essentially performs this optimization automatically, over the entire program.

Something like...

    renderPage :: Haxl Html
    renderPage = renderHeader + renderBody + renderRightPane + renderFooter
This will travel as far as possible through all paths in the AST collecting all IO to be performed in the first round (which, once fetched, will unblock more of the AST, and the process repeats until we have an answer).

Re: Facebook open sources Haxl

#52

Post by untothebreach »

seanmcdirmid wrote: post #43
simonmar wrote: post #31

Here's our paper about the ideas behind Haxl: http://community.haskell.org/~simonmar/papers/haxl-icfp14.pd...

Somehow I think this should have been upvoted to the top.

Simon Marlow and Sean McDirmid in the same thread? Be still my heart!

Re: Facebook open sources Haxl

#53

Post by m0nastic »

fiatjaf wrote: post #48

http://hackage.haskell.org/package/haxl Why do Haskell libraries on Hackage doesn't come even with a single example, getting started, how to use, quick start, nothing, really, just function declarations? This scares Haskell newbies.

The "documentation" on Hackage is almost universally just the haddock-generated files (which is why it's mostly just function declarations and type signatures).

Most libraries list a "Home Page" that more often than not includes more useful documentation (Haxl's, for example, has the things you've mentioned).

I concur, that most of the time, the documentation on Hackage isn't really sufficient, but I've found that for the most part I just use it to find the homepage, and then go there to read the actual documentation.

I agree that it would be nice if everything was all in one place.

Re: Facebook open sources Haxl

#54

Post by fnord_murks »

lbrandy wrote: post #34
tveita wrote: post #30

Earlier quoted context omitted.

I'm curious how this is executed. Is it like a query engine, where you work with the entire query up-front, apply transforms and build a query plan? Or is it more like an event loop, where you run as far as you can until the code blocks on IO, batch up and send all the pending IO requests, and run further when the tasks you're blocked on resolve?

Part of the beauty is that the actual way IO (note: in this version, IO here means 'reads from the network', almost always) is scheduled is abstracted away such that we could go with either approach w/o impacting client code. That said, the way it currently works is more like the first. You can think of the entire haxl run (program) as an AST that is given to the execution. It expands as much of the AST as possible (…

Have a look at the SQLTap service written by the guys from DaWanda.com (https://github.com/paulasmuth/sqltap). It does basically exactly that for SQL queries but is implemented as a standalone Java/Scala SQL proxy server.

Re: Facebook open sources Haxl

#55

Post by fnord_murks »

EGreg wrote: post #21

How do you guys batch requests in PHP? You don't, right? So this is an intermediate layer basically, and it sends requests every few millisecods and waits to batch things in between?

If you want to have similar query parallelization magic for PHP+SQL have a look at the standalone SQLTap service written by the guys from DaWanda.com (https://github.com/paulasmuth/sqltap).

Re: Facebook open sources Haxl

#56

Post by HannibalLecter »

lbrandy wrote: post #5

Hi. I'm one of the engineers who has worked on this so if anyone has any specific questions I can help answer and/or get someone to answer. As said by @nbm, we also have a blog post up: https://code.facebook.com/posts/302060973291128/open-sourcin... .

Will you comment as to whether FB has developed something superior to this or not?

Re: Facebook open sources Haxl

#57

Post by imanaccount »

msie wrote: post #50
thoughtpolice wrote: post #46

Earlier quoted context omitted.

To be completely honest, the namespace/module situation with Haskell could certainly be a _ton_ better, but after 8 years of it I can't ever remember a time when it was ever at the top of my mind as game-breaking. Occasionally quite annoying? Yes, most definitely. But I'd say there are more many more annoying things day to day, and in any case, it is certainly a tradeoff I'll put up with for the returns. That said, G…

Thanks! If it's not a pain in regular Haskell work then I'm relieved. Perhaps other data types are more prevalent in Haskell than records?

I find records are used most heavily in web development, where you are pretty much just shuffling data from browsers to databases and the other way around. But even there the field name thing doesn't pose much of a problem, I prefer defining my records in the module that handles the functions for it, so there's no issue with conflicting names anyways. I found the fact that 'id' is a standard library function to be a bigger minor annoyance.

Re: Facebook open sources Haxl

#58

Post by imanaccount »

fiatjaf wrote: post #48

http://hackage.haskell.org/package/haxl Why do Haskell libraries on Hackage doesn't come even with a single example, getting started, how to use, quick start, nothing, really, just function declarations? This scares Haskell newbies.

You mean "this library" not "Haskell libraries". Some libraries have real documentation (either on hackage or off) to go with the API reference.

http://hackage.haskell.org/package/pipes-4.1.2/docs/Pipes-Tu...

http://hackage.haskell.org/package/aeson-0.7.0.6/docs/Data-A...

Re: Facebook open sources Haxl

#59

Post by psibi »

tormeh wrote: post #37
coolsunglasses wrote: post #27

Earlier quoted context omitted.

Anybody that's made a serious go of FP in Scala (applicative, monad, etc) has found that it's not worth the hassle. It fights you the whole way.

Scalaz?

I think Scalaz is a pathway to Haskell. When people start loving Scalaz, they just happen to migrate to Haskell.

Re: Facebook open sources Haxl

#60

Post by polskibus »

How does the functionality of Haxl differ from a mature ORM system? I'm thinking about .NET Entity Framework + LINQ in particular since it not only does the mapping but also assists in query generation, scheduling.
Post reply on HN