Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

151–160 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#151

Post by jeswin »

sph wrote: post #64
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

> ... with 1/50th the effort of any other language.

To junior developers reading this, such statements are never true.

Re: Elixir saves Pinterest $2M a year in server costs

#152

Post by [deleted] »

sph wrote: post #114
_m8fo wrote: post #100

Earlier quoted context omitted.

[flagged]

Async/await and any module won't save you from global state, data races, and the fact that you're running on an imperative language with mutable state. Additionally, the ergonomics are not the same, so even if you could replicate the BEAM in Node or any other language, you'd have to be a masochist to do it. Lastly, the concurrency are primitives to the entire runtime, not a set of external libraries maintained by who…

[deleted]

Re: Elixir saves Pinterest $2M a year in server costs

#153

Post by acdha »

klohto wrote: post #26
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

That’s exactly the thing? Why would you bother optimizing the code, looking for overallocations, leakage, tweak parameters, when you can just take a friendlier language for the same benefits.

I think the question is whether they significantly changed the architecture at the same time. For example, reading the description of the Python migration sounds like they applied a lot of experience which would have benefited any language, and micro-optimizations like what you described would have been a rounding error on those larger changes:

https://medium.com/pinterest-engineering/fighting-spam-with-...

Re: Elixir saves Pinterest $2M a year in server costs

#154

Post by sharts »

Couldn't they have just moved their python to Cython / Pyrex? Or changed their arch and kept the tech stack the same?

I wonder how much time and team hours were spent learning the new framework and it's caveats vs if they'd just spent a little time optimizing their existing stack.

All too often common with non-perf folks making sweeping changes without rigorously measuring along the way. I guess it makes for good headlines.

Re: Elixir saves Pinterest $2M a year in server costs

#155

Post by andrewstuart »

200 servers running Python.

I bet it was an architecture problem more than a language problem.

I’d be expecting to leave most of that in place and fix the hotspots that are dragging performance down. Maybe some compiled code in certain places.

Almost certainly a detailed analysis would have yielded 20 things that could be tuned.

Posts like this are almost something to be ashamed of “we rewrote an entire subsystem because we wanted to use a language we like”. That’s really failing in your responsibility to the company.

Re: Elixir saves Pinterest $2M a year in server costs

#157

Post by jatins »

kramerger wrote: post #111

> José: Why did you choose Elixir? > Steve: We chose Elixir because we were looking for a system that was easy for programmers to understand and could take better advantage of our servers. I was intrigued at Elixir’s combination of friendly syntax, powerful metaprogramming features, and incorporation of the Actor model. Am I wrong or is this guy in some sort of bubble where only functional languages are taught?

I like Elixir as much as any other guy but I am not going to chose it for my team because it was "easy for programmers to understand"

If you wanted something that was easy you'd go got Golang or Java, not a language without types.

Lots of reasons to love Elixir but this doesn't sound like one

Re: Elixir saves Pinterest $2M a year in server costs

#158

Post by Coryodaniel »

m1keil wrote: post #118
Coryodaniel wrote: post #105

Earlier quoted context omitted.

waves

Haha, Hi Cory, I would really like if 5 years after that article and blog posts if you are still in the Elixir ecosystem?

Yep. Massdriver [0] is built on elixir and golang.

All user facing APIs and cloud provisioning state is written in elixir and runs on pretty much just fumes. Cloud systems interaction is written in golang because cloud stuff.

Email is in profile if you’d like to chat. Always happy to talk elixir and golang

0: https://massdriver.cloud

Re: Elixir saves Pinterest $2M a year in server costs

#160

Post by maccard »

intelVISA wrote: post #119
sph wrote: post #64

Earlier quoted context omitted.

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

Those naive sequential services don't really exist in prod though. It would be a dark day to discover my AWS t1.337metal was blocking 63 cores on I/O when nearly every modern lang has a wealth of async functionalities just awaiting to be exploited.

Oh man, you're in for a dark day.
Post reply on HN