Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fighting spam with Haskell

code.facebook.com

Post reply on HN
Print view View original

11–20 of 98 posts

Re: Fighting spam with Haskell

#11

Post by haxeny1 »

wz1000 wrote: post #9
Lewisham wrote: post #8

Earlier quoted context omitted.

Did you use a particular framework? I've been greatly enjoying my 4th attempt at learning Haskell (I think only now am I seasoned enough to really get it ), and my general proof that I know something and can use it usefully is to develop a web app (as all apps are now anyways). However, I didn't find anything which looked like it had majority community buy-in unlike Rails/Sinatra/Flask et al.

Yesod is the most actively developed and popular web framework for Haskell.

For my side project, I use yesod/websockets for the server and haxe for the client. I feel ok with that choice instead using hpaste. Though eventually I would like to see a complete haskell stack. @Lewissham: about the fourth attempt, it just so happens that this was my third attempt at getting something that was right for me as well as maintaining the typesafety. I do realize that code is still not as seamless as I would like. It still lets me understand the overall roadmap that the code needs to take to be usable/maintainable etc.

Re: Fighting spam with Haskell

#12

Post by mmarx »

wz1000 wrote: post #6

How does the hot-swapping work? The only way I had seen of making this happen is what xmonad does. I'm assuming this is radically different from that.

What xmonad does actually isn't hotswapping, but exec'ing the new code and simply passing the current state along. Instead they are using GHC's runtime linker to load/unload modules dynamically, in the same fashion as ghci does.

Re: Fighting spam with Haskell

#13

Post by Lewisham »

fokz wrote: post #4

I am under the impression that a large part of engineering effort at established companies go into porting existing components to a deemed to be more appropriate language for that task. Is it plain impossible to pick the best fit language without implementing a solution in the first place and fleshing out the requirements and challenges that specific to the problem space? Or do the problems evolve fast enough that no…

Bit of Column A, bit of Column B.

All software tends to unmaintainability, even if you don't touch it (libraries change/deprecate etc). Given enough time, the problems that you are experiencing, say, memory allocation in C, get solved by something else, such as garbage collectors, and the amount of time it takes to solve the headache in what you've got is longer than just rewriting into the new language.

Re: Fighting spam with Haskell

#14

Post by HugoDaniel »

This is an amazing effort, implementing ApplicativeDo and using Haxl for automatic batching and concurrency, doing code hot-swap in a compiled language, developing per-request automatic memoization, finding a aeson performance bug, translating C++ from/to haskell to do partial marshalling of data, implementing allocation limits to ghc threads, creating a shared c++ library to bundle the c++ dependencies in ghci for interactive coding, killing two ghc bugs, and more... and in the end producing a reliable scalable solution.

ouch!

Re: Fighting spam with Haskell

#15

Post by j_m_b »

|We implemented automatic memoization of top-level computations using a source-to-source translator. This is particularly beneficial in our use-case where multiple policies can refer to the same shared value, and we want to compute it only once. Note, this is per-request memoization rather than global memoization, which lazy evaluation already provides.

I would like to know more about this. What is a request exactly? An API call? If so, when an existing policy is changed, do the memoization tables have to change as well? How are the memoization tables shared? If this is running on a cluster, I would imagine that lookups in a memoization table could be a bottleneck to performance.

Re: Fighting spam with Haskell

#16

Post by dasmoth »

Damn you FaceBook.

I dislike the underlying premise, the adverts, and (especially) the "real names" policy.

But... between great bits of Open Source like React, cool infrastructure projects like this, and a technical culture which seems a whole lot more open than many other big companies, it's getting kind-of hard to go on hating. Walk back a bit from the obsession with open plan offices, and I might just cave...

Re: Fighting spam with Haskell

#17

Post by r0naa »

dasmoth wrote: post #16

Damn you FaceBook. I dislike the underlying premise, the adverts, and (especially) the "real names" policy. But... between great bits of Open Source like React, cool infrastructure projects like this, and a technical culture which seems a whole lot more open than many other big companies, it's getting kind-of hard to go on hating. Walk back a bit from the obsession with open plan offices, and I might just cave...

> But... between great bits of Open Source like React, cool infrastructure projects like this, and a technical culture which seems a whole lot more open than many other big companies, it's getting kind-of hard to go on hating. Walk back a bit from the obsession with open plan offices, and I might just cave...

Wholeheartedly agree with you comment, Facebook's engineering blog is truly humbling.

There is not a day where I do not try to think of something that would both make the Internet more decentralized, anonymous and secure while being extremely profitable. The latter is crucial because capitalism would make the change viral and being profitable would let me attract talent.

In short, make something that people want AND that promotes values I believe in.

Re: Fighting spam with Haskell

#18

Post by agentgt »

wz1000 wrote: post #6

How does the hot-swapping work? The only way I had seen of making this happen is what xmonad does. I'm assuming this is radically different from that.

I was wondering that myself. Maybe they will release code examples on how that works. I'm only slightly familiar with the Haskell ecosystem but I have also wondered if it has a plugin like system (e.g. OSGI for Java).

Re: Fighting spam with Haskell

#19

Post by vezzy-fnord »

Loading and unloading code currently uses GHC's built-in runtime linker, although in principle, we could use the system dynamic linker.

That they could use the system dynamic linker makes me think they're using some form of relatively basic dlopen/dlsym/call-method procedure, or something along those lines. That's fine, though the use of "hotswapping" evokes the image of some more elaborate DSU mechanism.

Re: Fighting spam with Haskell

#20

Post by simonmar »

j_m_b wrote: post #15

|We implemented automatic memoization of top-level computations using a source-to-source translator. This is particularly beneficial in our use-case where multiple policies can refer to the same shared value, and we want to compute it only once. Note, this is per-request memoization rather than global memoization, which lazy evaluation already provides. I would like to know more about this. What is a request exactly?…

For example, let's say that one of the things you want to compute is the number of friends of the current user. This value is used all over the codebase, but it only makes sense in the context of the current request (because every request has a different idea of "the current user"). So this is a memoized value, even though in the language it looks like a top-level expression.

Memoization only stores results during a request. It starts empty at the beginning of the request and is discarded at the end, and it is not shared with any other requests. It's just a map that's passed around (inside the monad) during a request.

Post reply on HN