Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

411–420 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#411

Post by throwaway894345 »

DoesntMatter22 wrote: post #396
fidotron wrote: post #254

Earlier quoted context omitted.

> If you are not willing to try it, that's fair. Say that. Claiming you know stuff about the ecosystem while a guy who is there every day comes across as... strange. Biased. And not arguing in good faith. :( I am not willing to drag others, such as those that wrote the repos, into a technical discussion with people out to act as you are.

The guy you are responding to was completely calm and reasonable. Didn't say anything attacking or otherwise. I'm not sure why you are seemingly trying to cast him (and the Beam) in such a bad light, with seemingly no reason to back it up.

Calling someone “biased” and “acting in bad faith” is a personal attack and violates this site’s rules. People get rate limited for far less on this site.

Re: Elixir saves Pinterest $2M a year in server costs

#412

Post by OkayPhysicist »

eropple wrote: post #311
Name_Chawps wrote: post #310

Earlier quoted context omitted.

I'm a junior Elixir dev. I write concurrent code without much extra effort compared to non-concurrent code. Elixir makes it easy.

You can do that easily in modern Java--even for older JVMs, tools like Netty and later Vertx have been around forever. Or in Node, even more easily. Elixir/BEAM do have some benefits that are worth considering for many projects. But they absolutely are not special in this regard, and that's the junior-developer trap about which the person to whom you replied was referring.

You may be enamored with the "nothing new under the sun" idea that "Turing complete is Turing complete, anything you can do in Python you can do in Brainfuck" but as someone who has written code professionally in about a dozen languages, no, you can't just as easily get the same kind of parallelism out of Java as you can Elixir. To assert otherwise is factually false.

Is it possible to get to the same result? Yes. It is not, however, anywhere in Elixir's ballpark of "easily". Do not discount the power of language-level, not just support, but encouragement. Especially when you're working with junior devs, if "the right thing" and "the thing the language wants you to write" are not in alignment, everything is much, much harder. Erlang and Elixir actively encourage easily parallelized code. Java activity encourages tangles of objects.

Re: Elixir saves Pinterest $2M a year in server costs

#413

Post by OkayPhysicist »

cutler wrote: post #280
zaphar wrote: post #238

Earlier quoted context omitted.

The go runtime has similar capabilities as the BEAM runtime when it comes to concurrent workloads. Go has the benefit of being a typesafe compiled language which gives it speed benefits. But using either one of them instead of Java is probably going to be a huge win for most teams on concurrent workloads.

> But using either one of them instead of Java is probably going to be a huge win for most teams on concurrent workloads Not for long. Something is looming on the horizon in Java-land.

Something's always looming over Java-land: The dark shadow of Oracle, watching over everything with it's flaming eye.

Re: Elixir saves Pinterest $2M a year in server costs

#414

Post by OkayPhysicist »

elcritch wrote: post #56
chii wrote: post #37

Earlier quoted context omitted.

> it's very likely that a Java-to-Java rewrite would have helped go down too. it depends. The elixir programming paradigm might be one where you are able to more easily write efficient, but still highly concurrent code, where as it would take more work to do the same in java.

Elixir / Erlang does open up a lot of paradigms. However it'll be interesting to see how Java fairs with its new virtual threads. There was already Akka so it should happen fairly quickly. Still I'd prefer Elixir. The BEAM VM just runs lighter.

If it's cludged on like Java's attempt at supporting basic functional programming, it's guaranteed to painful regardless.

Re: Elixir saves Pinterest $2M a year in server costs

#415

Post by OkayPhysicist »

Cthulhu_ wrote: post #68
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

I think it's more important to look at the re-architecting than the different language. Second, I think certain architectures - like the actor model described in the article - work better and more intuitive if you use a different language. That said, I'm sure a 2x performance improvement could've been done in Java as well if they did a re-architecture. They could also have made a lateral movement and go to a differen…

> certain architectures... work better and more intuitive if you use a different language.

This is the key point that people miss when pretending that languages are interchangeable. The entire point of making a programming language is to make certain types of ways to solve problems easier to express. This constitutes a language's "pretty path". By providing such pretty paths, languages necessarily make less desirable paths, which will be painful to slog through.

If you try writing a functional pipeline in Java, you're going to have a much worse time than doing the same in Elixir. If you try to do Object-Oriented class towers in Scheme, it's going to be painful. Etc, Etc. You can write a Rust program and a C program that compile to the exact same binary, but I can put a whole stack of cash on which one's going to be easier.

Re: Elixir saves Pinterest $2M a year in server costs

#416

Post by hinkley »

coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

How many man hours did it take to achieve that?

We just did a Prometheus migration that I suspect will take us 5 years to break even on the development effort investment. And I'm not even counting opportunity costs, which were immeasurable.

I like Elixir and I want it to do well, but bad articles make that harder, not easier.

Re: Elixir saves Pinterest $2M a year in server costs

#417

Post by hinkley »

Cthulhu_ wrote: post #76
sergioisidoro wrote: post #48

I am very skeptical about this. The title makes it look like it's all about Elixir, but there seems to have been a fair amount of re-architecturing. "One of the systems that ran on 200 Python servers now runs on four Elixir servers" This alone is a major telltale.

For sure; Elixir comes with a whole new architecture as well, and they COULD have gained a significant performance improvement if they rewrote / re-architected it in Python. However, could they have done a 50x performance improvement in Python? And what about the other numbers, like speed and concurrency? That said, I'm confident they crunched the numbers and did the tradeoffs; after all, adding another language and/…

They would have had to create a bad implementation of half of Erlang to accomplish it. Or you could just fuckin' use Elixir.

Re: Elixir saves Pinterest $2M a year in server costs

#418

Post by DoesntMatter22 »

throwaway894345 wrote: post #411
DoesntMatter22 wrote: post #396

Earlier quoted context omitted.

The guy you are responding to was completely calm and reasonable. Didn't say anything attacking or otherwise. I'm not sure why you are seemingly trying to cast him (and the Beam) in such a bad light, with seemingly no reason to back it up.

Calling someone “biased” and “acting in bad faith” is a personal attack and violates this site’s rules. People get rate limited for far less on this site.

He didn't call him that at all. He said him being unwilling to explain his points and instead just making claims comes across that way.

He at no time called him biased or said he was directly acting in bad faith

Re: Elixir saves Pinterest $2M a year in server costs

#419

Post by DoesntMatter22 »

fidotron wrote: post #405
DoesntMatter22 wrote: post #396

Earlier quoted context omitted.

The guy you are responding to was completely calm and reasonable. Didn't say anything attacking or otherwise. I'm not sure why you are seemingly trying to cast him (and the Beam) in such a bad light, with seemingly no reason to back it up.

He is adopting the missionary tactic of feigning a desire for discussion when it is just veiled evangelism.

He seemed to be asking you to back up your claims and you seem to be just saying claiming that you don't need to and he is the problem...

Re: Elixir saves Pinterest $2M a year in server costs

#420

Post by DoesntMatter22 »

winter_blue wrote: post #399
DoesntMatter22 wrote: post #375

Earlier quoted context omitted.

For the vast majority of projects this makes no difference. If you are at Facebook scale? Sure. But then, you do what they did, write a VM to speed things up. Dynamic typing doesn't cost much more money on average and thankfully the cost of energy itself is a motivation for companies to do rewrites. If they are paying a lot in server costs and electricity then they do typically rewrite to reduce the amount of servers…

> running at a profit and getting to market quickly which dynamic languages do extremely well This (or similar variants of this) is an assertion that's commonly made about dynamically-typed languages, but I don't think they hold any water. Less readable code (due to the lack of types) makes it a lot harder to add new features, and harder to debug code as well. Several years ago, I briefly worked on a fairly large cod…

The main argument here was about the energy costs. But it sounds as though you don't like dynamically typed languages. That's fine.

Facebook, twitter and plenty of of billion dollar businesses were built with dynamic languages and many would argue they may not have even existed if they were written in staticly typed languages due to the slower up front time expenditure.

I like staticly types languages for large projects but enjoy thr development speed of dynamic ones. If you don't like dynamic that's completely fine

Post reply on HN