Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

161–170 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#161

Post by __alexs »

Cthulhu_ wrote: post #76
sergioisidoro wrote: post #48

I am very skeptical about this. The title makes it look like it's all about Elixir, but there seems to have been a fair amount of re-architecturing. "One of the systems that ran on 200 Python servers now runs on four Elixir servers" This alone is a major telltale.

For sure; Elixir comes with a whole new architecture as well, and they COULD have gained a significant performance improvement if they rewrote / re-architected it in Python. However, could they have done a 50x performance improvement in Python? And what about the other numbers, like speed and concurrency? That said, I'm confident they crunched the numbers and did the tradeoffs; after all, adding another language and/…

> I'm confident they crunched the numbers and did the tradeoffs;

Here is the math they did.

Pros: Looks good in my promotion packet.

Cons: Need a bigger garage to fit all of these new sports cars.

Re: Elixir saves Pinterest $2M a year in server costs

#162

Post by acdha »

andersa wrote: post #115
coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

It's really strange to me how people continue to build services using python, knowing it is 100 times slower than appropriate languages, and then get surprised by it being 100 times slower, so they eventually rewrite it.

It’s extremely rare for a system to be 10x slower, much less 100x, and developer productivity is huge. When you see huge numbers being tossed around for an entire system, they almost always mean “our first architecture wasn’t right for the problem” and the question to ask is how much time it would have taken the same team to discover the correct shape of the problem with the other candidates.

Re: Elixir saves Pinterest $2M a year in server costs

#163

Post by Incolotopo »

Coryodaniel wrote: post #135
samber wrote: post #86

They just stopped coding lambda... This is not a matter of a language. Lambda is costly.

Author of the “Lambda” piece [0], all of the cost savings I discussed came from API Gateway alone. The lambdas were negligible. That being said, I don’t think we could have processed the traffic more efficiently in another language. We were processing 12M requests per hour. You could run the entire thing on 2 vCPU we “over provisioned the crap out of it” by running it on 4 pods w 2 vCPU as the upper request limit. >…

Its still not very easy to understand if this is now good or not.

3-5k requests per second is not that much and highly depends on what you actually do.

A lot of things can be done just with nginx. nginx is very fast.

Re: Elixir saves Pinterest $2M a year in server costs

#164

Post by mgaunard »

sph wrote: post #64
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

Using more threads usually makes I/O-bound tasks slower, not faster, unless they're working with different devices or resources.

What you need to be fast is a systems engineer.

Re: Elixir saves Pinterest $2M a year in server costs

#165

Post by Incolotopo »

varispeed wrote: post #143

Surely this means they could donate at least $1M a year to the Elixir team?

yes but besides a somehow snarky comment, it doesn't make sense from a company perspective.

This wasn't done to have mone to donate to Elixir but for revenue.

And besides that, you don't even know if they do

Re: Elixir saves Pinterest $2M a year in server costs

#166

Post by amelius »

giancarlostoro wrote: post #133
kramerger wrote: post #113

Earlier quoted context omitted.

That perfectly describes my usage of Pinterest. People have written browser extensions to remove Pinterest from search results, as it is almost always a dead-end

Maybe to you, but my wife uses Pinterest a ton. It is where a lot of the women I know go to for ideas for just about everything, from house decor to even the ideas for our wedding. Pinterest is useful just not for us nerdy guys. I am not sure why Google keeps it though or how they benefit, unless Pinterest uses AdSense exclusively, then one can determine that its some sort of partnership. You would think Google would…

> It is where a lot of the women I know go to for ideas for just about everything, from house decor to even the ideas for our wedding.

So much for the idea that men are more "vision-oriented" than women.

Re: Elixir saves Pinterest $2M a year in server costs

#167

Post by knewter »

jeswin wrote: post #151
sph wrote: post #64

Earlier quoted context omitted.

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

> ... with 1/50th the effort of any other language. To junior developers reading this, such statements are never true.

Though if people consistently over decades sing a language's praises on a single point consistently like they do the beam on this point, it's usually not without merit

Re: Elixir saves Pinterest $2M a year in server costs

#168

Post by Incolotopo »

0xpgm wrote: post #29
lambda_dn wrote: post #9

They could have just used different Java/Python libraries and changed the architecture for the same result. The choice of a Programming Language alone has minimal impact on performance at that scale.

Can you give a concrete example where such a result was achieved by only switching libraries and modifying architecture?

I mean there are plenty of examples in the wild left and right.

Have you ever seen basic db optimization? Alone in my companies people were just using stuff wrong.

Performance and Architecture are a after thought in our industry. A normal developer doesn't think about it.

There was one query in my company which was running in one region slower than in another one and there was also an explain statment available. No one looked at the explain statement and thought "huh why does this simple select use so much memory". People weretrying to see why the regions themselfs were different not what the problem with the query was.

Re: Elixir saves Pinterest $2M a year in server costs

#169

Post by bradleykingz »

giancarlostoro wrote: post #133
kramerger wrote: post #113

Earlier quoted context omitted.

That perfectly describes my usage of Pinterest. People have written browser extensions to remove Pinterest from search results, as it is almost always a dead-end

Maybe to you, but my wife uses Pinterest a ton. It is where a lot of the women I know go to for ideas for just about everything, from house decor to even the ideas for our wedding. Pinterest is useful just not for us nerdy guys. I am not sure why Google keeps it though or how they benefit, unless Pinterest uses AdSense exclusively, then one can determine that its some sort of partnership. You would think Google would…

It's one thing to use Pinterest intentionally, a whole other to have it shoved down your throat.

Re: Elixir saves Pinterest $2M a year in server costs

#170

Post by imglorp »

masklinn wrote: post #129
_m8fo wrote: post #100

Earlier quoted context omitted.

[flagged]

It’s not even remotely similar. Node’s cluster is just bog-standard OS subprocesses running their own event loop. To spread work over multiple cores with the cluster (or even worker-threads) modules you have to do so explicitely and manually. It’s essentially the same model you get with pthreads, or java, or python. BEAM is a completely different model, the “processes” are internal tasks, which the runtime will sched…

At the Abstractions conference in Pittsburgh in 2016, Joe Armstrong was hanging out in the hallway with his swag bag, just a regular engineer complaining about Jira and his manager (he had no interest in managing) and asking people's opinion about the schedule and lunch places. We were looking at the program for the next sessions and someone said there's a talk about ideas for adding concurrency features to Node, and we said great, lets go, and a few of us went to go stand in the back of that one.

On a number of points the presenter was proposing, like message passing, immutable structures, and process tree management, the presenter would say, "but Erlang's had this feature for many years..." and the room would laugh and turn around and acknowledge Joe. He was modest but the validation must have been nice.

Post reply on HN