Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

61–70 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#61

Post by abrookewood »

te_chris wrote: post #52
DoesntMatter22 wrote: post #46

Earlier quoted context omitted.

Elixir is slower than plain PHP according to the techempower benchmarks. I'm not even sure how that's possible but it is. By like a factor of 2 iirc. I'm not sure how elixir is that slow since it's compiled.

PHP is fast these days raw-speed wise. Concurrency and parallelism matter more than speed as they translate directly to utilisation.

Yes, calling something Fast can mean many different things. Erlang/Elixir are not 'fast' at number crunching, but they are very good at concurrency.

Re: Elixir saves Pinterest $2M a year in server costs

#62

Post by xk_id »

interactivecode wrote: post #15
redocneknurd wrote: post #7

Pinterest - ah that app that has hijacked all images in google search result. Created an account once but has not really used it.

While the google images thing is kinda shitty. Pinterest is really really good at image search and image recommendations.

It's actually one of my favourite apps, after using it for years. I'm not even joking. Their feed is incredibly good.

Re: Elixir saves Pinterest $2M a year in server costs

#63

Post by fergie »

Anecdotally, I am beginning to hear more and more about organisations moving away from high level cloud infrastructure (such as lambda and cloud gateway) and going back to plain old virtual servers (like EC2), or even on-prem. Often the cost of supposedly "cheap" cloud environments is WAY more than you might expect and all booked as operational rather than capital expenditure (the latter being often preferable to shareholders).

Re: Elixir saves Pinterest $2M a year in server costs

#64

Post by sph »

threeseed wrote: post #17
coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

The advantages of Elixir are not performance-related.

There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.

We live in a world with 32+ core CPUs. If your load is not spread uniformly all over those cores, you're losing a ton of performance. Handling requests over separate threads, like 99% of languages do, still isn't enough if all the business logic runs on the same thread.

I'm currently writing a web crawler in Elixir, and it is easier to design it so every request is done and processed in parallel, than to write a naive sequential one you'd do in any other language in half a day.

Re: Elixir saves Pinterest $2M a year in server costs

#65

Post by elcritch »

laserbeam wrote: post #35
vb-8448 wrote: post #21

I wonder how much of the savings comes from the refactoring and how much from the difference between the speed of python and elixir.

It's not unusual to gain 50-100x performance gains on code when moving from python to a compiled language without much refactoring. Dropping from 200 to 4 servers is well within the lines of performance gains one can expect. Yes, those numbers are for code, network speeds are not directly impacted the same way. But again, does not sound unreasonable.

While Elixir is technically a compiled language, it's only compiled in the same sense that Python compiles to byte codes.

Re: Elixir saves Pinterest $2M a year in server costs

#66

Post by TekMol »

My problem with Elixir is that the syntax is too noisy:

    defmodule Math do
        def sum(a, b) do
            a+b
        end
    end
Compare that with Python:

    def sum(a, b):
        return a+b
Having the name of the file be the name of the module and whitespace defining the blocks is the perfect approach for me.

Re: Elixir saves Pinterest $2M a year in server costs

#67

Post by itronitron »

xk_id wrote: post #60
dazc wrote: post #53

Earlier quoted context omitted.

Google are returning results for image search without the overhead cost. This is the only logical reason I can think of why Pinterest still exists.

I was never sure where all the hate is coming from. Pinterest is actually one of my favourite apps, that I use regularly – and I use very few apps. I actually really love it and after years I still find new uses for it. As far as I can tell, they have a pretty loyal user base.

The hate stems from Pinterest requiring an account, which they didn't start out requiring, just like Instagram which I also don't visit.

Re: Elixir saves Pinterest $2M a year in server costs

#68

Post by Cthulhu_ »

threeseed wrote: post #17
coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

I think it's more important to look at the re-architecting than the different language. Second, I think certain architectures - like the actor model described in the article - work better and more intuitive if you use a different language.

That said, I'm sure a 2x performance improvement could've been done in Java as well if they did a re-architecture. They could also have made a lateral movement and go to a different JVM language, like Scala that also has an actor concurrency model + accompanying syntax.

Re: Elixir saves Pinterest $2M a year in server costs

#69

Post by magicCat »

diarrhea wrote: post #50
manmal wrote: post #41

Earlier quoted context omitted.

Wouldn't you also need a smaller ops team that way, further reducing your costs?

Managing 200 Python servers/nodes certainly sounds operationally challenging.

i disagree, not much more then 2 servers.. you should automate it fully for documentation purposes alone at this scale & test failover regulary

if you do this with 4 or 200 machines, does not make a big difference.

Post reply on HN