Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

301–310 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#301

Post by moomoo11 »

kramerger wrote: post #131
capableweb wrote: post #120

Earlier quoted context omitted.

You seem to be wrong, at least based on the quote you chose to use here, as Steve doesn't even mention anything related to functional programming?

Elixir is a functional language for the Erlang VM. Most C++/Java/Rust/... programmers wpuld have hard time understanding it. But if you are in a bubble where everyone uses Haskell and talks about Monads, then OPs statement may be valid

Don’t forget having to know beam and erlang especially if post junior/intern level where it’s more than just “it looks like ruby”

Re: Elixir saves Pinterest $2M a year in server costs

#302

Post by manmal »

magicCat wrote: post #69
diarrhea wrote: post #50

Earlier quoted context omitted.

Managing 200 Python servers/nodes certainly sounds operationally challenging.

i disagree, not much more then 2 servers.. you should automate it fully for documentation purposes alone at this scale & test failover regulary if you do this with 4 or 200 machines, does not make a big difference.

Running 200 servers will force you to automate everything and also handle (relatively) rare edge cases, meaning added complexity, huge up front dev costs, and a continuous dev effort just to stay afloat. Work has to be balanced in a clever way, making tradeoffs that you otherwise wouldn't need to make. The mental model shifts from individual servers to a whole ecosystem that shows emergent behavior (= things break in a totally new way).

Maybe some of that also applies to 4 servers, but to a way lesser extent.

Re: Elixir saves Pinterest $2M a year in server costs

#303

Post by no_wizard »

princevegeta89 wrote: post #294
coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

The problem with Elixir is that it is such a foreign language to most of the junior developers and a radical shift to dig into coming from object-oriented and other higher-level languages. This is worsened by the fact that there are not many jobs for Elixir in addition to Development Tooling, IDE Support. To someone who starts their job on an Elixir codebase, it is just not a smooth onboarding at all. While the perfo…

sounds like a feature not a bug. Poor onboarding is a cultural problem in my experience, not really a technology one.

When you get junior (or even non junior) developers onboarded in a new language, you have a unique opportunity to break them of bad habits and expand horizons.

Yes, there is a cost to it as it extends in the short term the time it takes to get developers ramped up, however the long tail payoff is huge

Re: Elixir saves Pinterest $2M a year in server costs

#304

Post by KomoD »

swarnie wrote: post #211
bradleykingz wrote: post #169

Earlier quoted context omitted.

It's one thing to use Pinterest intentionally, a whole other to have it shoved down your throat.

Where / How are you finding Pinterest links? I've just searched about two years of browser history and i haven't landed their once.

> Where / How are you finding Pinterest links?

Google, I google anything and they show up (ESPECIALLY IMAGES). Blacklist one domain in search? Don't worry they've got 200 other ones.

I ended up doing a wildcard blacklist with uBlacklist

Re: Elixir saves Pinterest $2M a year in server costs

#305

Post by tgv »

metaphor wrote: post #89
cebert wrote: post #57

Earlier quoted context omitted.

I also wonder what the carbon emission reduction would be from this change.

An even less significant rounding error; on the order of ~0.002% relative to AWS carbon footprint at best using FY23 forward net sales as a rough proxy.

Since there are really many companies and people using AWS, the effect would be noticeable if everyone managed such a decrease. Calling it "an even less significant rounding error" borders on bad faith.

Re: Elixir saves Pinterest $2M a year in server costs

#306

Post by panick21_ »

pdimitar wrote: post #196
miroljub wrote: post #181

Earlier quoted context omitted.

Even if we take this as a factor, there's still no reason why would a Go or Java or SBCL implementation be significantly worse than Elixir.

There is a reason: the BEAM is almost not prone to huge GC pauses. Bigger load results in every actor responding very slightly slower. Nothing else. Many other systems don't have this property. They fall down under pressure. Gosh, a huge chunk of HN is always so dismissive. At least read up a bit beforehand, man. The criticisms should be informed and benefit the readers, not only express a generic skepticism.

If and that's a very, very, very big if, the current open GC are really leading to to much pauses. Then you can go to Azul and buy a better VM and GC, further improving the performance compared to BEAM.

Re: Elixir saves Pinterest $2M a year in server costs

#307

Post by javanonymous »

neeleshs wrote: post #253
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

Yep. We got a 10x improvement on throughput for our backend runtime, which was in Java, by moving to a better architecture for performance hotspots.. using Java again. In a rewrite with a different design/architecture, that new design typically accounts for most gains, rather than language. A language may make some parts of that rewrite simpler.

I've gotten 100x improvement with no code change by just adding an index in the database table. An inexperienced developer might have blamed the database and insisted on moving to NoSQL because of "web scale". If they got the chance to rewrite it, they could have pointed to the performance increase as a proof that they were right.

Re: Elixir saves Pinterest $2M a year in server costs

#308

Post by princevegeta89 »

no_wizard wrote: post #303
princevegeta89 wrote: post #294

Earlier quoted context omitted.

The problem with Elixir is that it is such a foreign language to most of the junior developers and a radical shift to dig into coming from object-oriented and other higher-level languages. This is worsened by the fact that there are not many jobs for Elixir in addition to Development Tooling, IDE Support. To someone who starts their job on an Elixir codebase, it is just not a smooth onboarding at all. While the perfo…

sounds like a feature not a bug. Poor onboarding is a cultural problem in my experience, not really a technology one. When you get junior (or even non junior) developers onboarded in a new language, you have a unique opportunity to break them of bad habits and expand horizons. Yes, there is a cost to it as it extends in the short term the time it takes to get developers ramped up, however the long tail payoff is huge

>>> the long tail payoff is huge

In what terms, exactly?

Re: Elixir saves Pinterest $2M a year in server costs

#309

Post by pessimizer »

dpkirchner wrote: post #248
bearjaws wrote: post #215

Earlier quoted context omitted.

They are referring to how they link out their images and SEO content that ultimately one will find unusable & frustrating. I have experienced this many times when looking for a specific image on Google Images, I will click on the link that goes to Pinterest only to find the image is not there.

IMO this is more a Google problem than a Pinterest problem. Google does the same thing with text search (returning results that have few if any of your search terms), possibly over relying on opengraph tags and other such metadata.

Pinterest did this on purpose, for traffic, and have maintained it for many years now. If either company saw it as a problem, it wouldn't be.

Re: Elixir saves Pinterest $2M a year in server costs

#310

Post by Name_Chawps »

jeswin wrote: post #151
sph wrote: post #64

Earlier quoted context omitted.

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

> ... with 1/50th the effort of any other language. To junior developers reading this, such statements are never true.

I'm a junior Elixir dev.

I write concurrent code without much extra effort compared to non-concurrent code. Elixir makes it easy.

Post reply on HN