Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fighting spam with Haskell

code.facebook.com

Post reply on HN
Print view View original

31–40 of 98 posts

Re: Fighting spam with Haskell

#31

Post by LukeHoersten »

zmanian wrote: post #28
LukeHoersten wrote: post #22

For those not involved in the Haskell community, Simon Marlow worked full time on the GHC compiler and specifically run-time system for many years. Along with Simon Peyton-Jones, he's huge in the Haskell world. Marlow also wrote the very excellent "Parallel and Concurrent Programming in Haskell" book. Facebook also employs Bryan O'Sullivan, an epic Haskell library writer (Aeson, Attoparsec, Text, Vector, and on and o…

Bos manages the internal tooling team at Facebook. It is little surprise that Facebook's internal tooling is acquiring a functional programming flavor.

Speaking of functional/Haskell network tooling, this is a really cool project: http://www.pfq.io

Re: Fighting spam with Haskell

#32

Post by LukeHoersten »

Periodic wrote: post #29
LukeHoersten wrote: post #22

For those not involved in the Haskell community, Simon Marlow worked full time on the GHC compiler and specifically run-time system for many years. Along with Simon Peyton-Jones, he's huge in the Haskell world. Marlow also wrote the very excellent "Parallel and Concurrent Programming in Haskell" book. Facebook also employs Bryan O'Sullivan, an epic Haskell library writer (Aeson, Attoparsec, Text, Vector, and on and o…

It really shows what an investment they are willing to make in the language. They aren't just training up internal engineers, they're bringing in some of the best in the community. I'm hugely excited that Facebook is making this investment and giving some of the developments back to the community. There are many smaller companies that would be interested in Haskell but don't have the resources or expertise to tackle…

One thing's for sure, Facebook staffing did an amazing job hiring Haskell devs ;-)

Re: Fighting spam with Haskell

#33

Post by Scarbutt »

edwinnathaniel wrote: post #24

I learned a few things from this post outside the usual "technical" explanations: 1. They have CORE Haskell contributors on their payroll to deliver this type of project (what this mean is that no... Haskell isn't any better than other language, it's just that they have people who know Haskell very very deep to the compiler level...) 2. In-house custom language eventually does not scale (the EOL is much much much sho…

What you should ask yourself then is, why did they hire core Haskell contributors?

Re: Fighting spam with Haskell

#34

Post by Filligree »

Periodic wrote: post #30
covi wrote: post #23

In the throughput graph, why does Haxl perform worse in the 3 most common request types?

I don't think the graph says which request types are more or less common. The ordering could be arbitrary. It would be interesting to know why some are slower. Perhaps they require very little processing and so the time becomes dominated by FFI transformations? It would be nice to know!

In fact, it appears to be ordered by performance. There's no other reason I can think of for that appearance.

Re: Fighting spam with Haskell

#35

Post by JonCoens »

Periodic wrote: post #30
covi wrote: post #23

In the throughput graph, why does Haxl perform worse in the 3 most common request types?

I don't think the graph says which request types are more or less common. The ordering could be arbitrary. It would be interesting to know why some are slower. Perhaps they require very little processing and so the time becomes dominated by FFI transformations? It would be nice to know!

One of the cases we found while performance profiling was a tradeoff between memory usage and computation completion. On certain requests FXL would use too much memory and be halted by the equivalent to AllocationLimits, while Haxl would happily plow on using less memory and complete the request. When looking at many of those requests in aggregate, the end result would have more successfully completed requests but with longer response times mixed in. Completing more requests was seen as a win over the apparent decrease in throughput.

Re: Fighting spam with Haskell

#36

Post by dasil003 »

edwinnathaniel wrote: post #24

I learned a few things from this post outside the usual "technical" explanations: 1. They have CORE Haskell contributors on their payroll to deliver this type of project (what this mean is that no... Haskell isn't any better than other language, it's just that they have people who know Haskell very very deep to the compiler level...) 2. In-house custom language eventually does not scale (the EOL is much much much sho…

> (what this mean is that no... Haskell isn't any better than other language, it's just that they have people who know Haskell very very deep to the compiler level...)

This is extremely presumptuous for a parenthetical. If this is what you believe it merits a comment all to itself, because I don't see how this follows in the least. To rephrase what you're claiming: if a language has a flaw than it can not be any better than any other language.

Re: Fighting spam with Haskell

#38

Post by bpyne »

fokz wrote: post #4

I am under the impression that a large part of engineering effort at established companies go into porting existing components to a deemed to be more appropriate language for that task. Is it plain impossible to pick the best fit language without implementing a solution in the first place and fleshing out the requirements and challenges that specific to the problem space? Or do the problems evolve fast enough that no…

It's tricky picking a "best fit language" when you're not sure at project inception what you're fitting it to. Say, for instance, that you're developing v. 1.0 of an application in a new application space. Your CEO catches wind that another company is looking to enter that same space. She is also adjusting requirements to find a business model that works. You would choose a development stack that allows rapid prototyping and refactoring.

Once you win user-share in the space the application starts having problems with scalability or debugging becomes an issue or a myriad of other problems that come with success. Now you need to consider writing parts (or possibly the whole application) in a development stack that allows greater scalability or has built-in semantics for core functionality in your application, thus allowing less reliance on 3rd party libraries, etc.

The other factor is that the team who authors an application may have a totally different skill set from the team who takes over the mature application. The authors may have been VB developers with good SQL skills. Your current team has more people who have dealt with Haskell in large-scale applications.

There is some parallel to home-building. A homeowner does a room conversion for more space, more privacy, etc. The builders of the home didn't install tracks and sliding-panel walls for better room configuration. At the time, they didn't see a need. The homeowner now has a need, so he goes through a much more expensive (relatively speaking) building effort.

Re: Fighting spam with Haskell

#39

Post by j_m_b »

simonmar wrote: post #20
j_m_b wrote: post #15

|We implemented automatic memoization of top-level computations using a source-to-source translator. This is particularly beneficial in our use-case where multiple policies can refer to the same shared value, and we want to compute it only once. Note, this is per-request memoization rather than global memoization, which lazy evaluation already provides. I would like to know more about this. What is a request exactly?…

For example, let's say that one of the things you want to compute is the number of friends of the current user. This value is used all over the codebase, but it only makes sense in the context of the current request (because every request has a different idea of "the current user"). So this is a memoized value, even though in the language it looks like a top-level expression. Memoization only stores results during a…

Thanks for the response. Just trying to expand my brain here =), so I have a followup question.

I always thought of memoization as storing the parameters to, and result of, a function call in a memotable. Doing some quick research, I came across this definition of memoization from NIST that sounds more general "Save (memoize) a computed answer for possible later reuse, rather than recomputing the answer." What I understand from what you said is that when a request is processed, it produces a map that is passed around for the duration of the request.

Something like:

Request -> (some processes) -> memoized map -> Policy Filters

How is the memoized map reused?

Re: Fighting spam with Haskell

#40

Post by wyager »

Lewisham wrote: post #8
seddona wrote: post #2

Thanks for the overview Simon, great to hear about the use of Haskell at scale. At CircuitHub we use Haskell to build our entire web app, Haskell is great for most tasks these days.

Did you use a particular framework? I've been greatly enjoying my 4th attempt at learning Haskell (I think only now am I seasoned enough to really get it ), and my general proof that I know something and can use it usefully is to develop a web app (as all apps are now anyways). However, I didn't find anything which looked like it had majority community buy-in unlike Rails/Sinatra/Flask et al.

Happstack, Snap, and Yesod are popular. I'm partial to Happstack because it doesn't use any wonky extensions like template Haskell. It's just a very straightforward, very fast webserver. It's also very agnostic of how you actually render your HTML, whether it's static serving or dynamic generation with Blaze-builder or something.
Post reply on HN