Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

81–90 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#81

I argue every project has to go through three phases - no idea what's wrong - No idea what's right - We know what was right, but the world has changed now.

I mean you're not wrong; they built working software in Python, solved a problem, scaled it, and learned about the problem it solved. Once they understood it, they could monitor it. Once they had numbers, they could look into improving it, taking their learnings on the problem and picking a different language / architecture to solve it.

Re: Elixir saves Pinterest $2M a year in server costs

#82

Earlier quoted context omitted.

Is this a cpu-heavy workload? If it's mostly io-heavy, I could see a gevent + pypy approach getting 50x better.

[flagged]

I don’t tend rewrite Python code any more often than is needed due to feature changes or occasionally refactoring to pay off some maintenance friction caused by design that has in practice turned out to be suboptimal.

What are you talking about?

Re: Elixir saves Pinterest $2M a year in server costs

#83
post #63

Anecdotally, I am beginning to hear more and more about organisations moving away from high level cloud infrastructure (such as lambda and cloud gateway) and going back to plain old virtual servers (like EC2), or even on-prem. Often the cost of supposedly "cheap" cloud environments is WAY more than you might expect and all booked as operational rather than capital expenditure (the latter being often preferable to sha…

I'm not sure why that would be too surprising to you. In the enterprise organization I've worked with over the past few decades IT strategy is always long term and always cost based. 10-15 years ago organizations moved from the basement to placing their owned hardware in rented racks in data centers that were run by 3'rd party organization because it was cheaper. Then Azure came along and made it sort of a "no-brainer" to move into Azure because you already had a lot of Microsoft products and Azure was cheap. Now with so many Azure price hikes and those 3'rd party data centers improving their business models, the pendulum is swinging away from Azure.

That doesn't mean that the move into Azure wasn't the right one at the time, or that it was more expensive than not going into Azure. It's simply that the market evolves.

Re: Elixir saves Pinterest $2M a year in server costs

#84

Compared to Python. No wonder.

Python is fine to build a first version; you should never pick a language or architecture because you THINK you may need its performance, that's cargo cult. Pinterest could not have predicted they needed 200 servers for this workload - and they probably didn't for a long time.

Re: Elixir saves Pinterest $2M a year in server costs

#85
post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

>Would be curious to know how they tried to optimise the Java stack

They say "The combined effect of better architecture and Elixir".

So the post is mostly useless.

Re: Elixir saves Pinterest $2M a year in server costs

#87

Earlier quoted context omitted.

To be honest the world at large doesn't have an anorexia problem but an obesity epidemy. 50 millions children under the age of 5 are already obese (and I think 93% of obese children shall stay obese their entire life ). So... Anorexia may be a hill to die on and it's not fun for parents of anorexic kids but priorities, priorities and priorities. The real eating disorder worldwide leads to obesity, not anorexia and I…

Though I think the original comment was in reference to teen-aged girls and not the world at large.

Clearly.

But the broader point beyond my specific snipe -- which is personal and driven by deep anger over a serious problem affecting me personally -- the broader issue is not only that Pinterest is on the whole an anti-social actor because of its garbage moderation and promotion of harmful content in its algorithms, it is more broadly a bad actor in the web ecosystem generally.

They hijack search results by scraping and stealing content, hiding the original link, and try to trap you in their circle of links.

In this day and age of people getting sued and slapped down for legitimate webscraping, it boggles my mind that Pinterest gets away with what they do.

On the whole a swamp of an unethical company. F*ck pinterest.

(But love that the commenter couldn't help but turn the thread into an underhanded comment against "fat people". Oh so typical.)

Re: Elixir saves Pinterest $2M a year in server costs

#88
post #64

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

[deleted]

Re: Elixir saves Pinterest $2M a year in server costs

#89
post #57

For context, $2MM ~= 0.3% of FY22 cost of revenue[1]. [1] https://www.sec.gov/Archives/edgar/data/1506293/000150629323...

I also wonder what the carbon emission reduction would be from this change.

An even less significant rounding error; on the order of ~0.002% relative to AWS carbon footprint at best using FY23 forward net sales as a rough proxy.

Re: Elixir saves Pinterest $2M a year in server costs

#90

Earlier quoted context omitted.

Is this a cpu-heavy workload? If it's mostly io-heavy, I could see a gevent + pypy approach getting 50x better.

[flagged]

In what planet?

This has absolutely no bearing to the Python experience. And I'm using the stuff for 20 years.

Post reply on HN