Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

391–400 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#391

Post by fidotron »

throwawaymaths wrote: post #378
fidotron wrote: post #366

Earlier quoted context omitted.

The underlying problem we had* was the rate work was being completed was lower than the rate requests were coming in which causes the mailboxes on the actors to grow indefinitely. In golang the approximate equivalent is a buffered channel that would start blocking because it has run out, but the beam will just keep those messages piling on to the queue as fast as possible until OOM. This is obviously a philosophical…

Why weren't your processes dropping messages? Also I think you can tell the VM to not allow the process to exceed a certain message size and trigger some sort of rate limiting or scaling out Edit: huh. I could swear the VM had memory limit options. Guess not. Time to rewrite it in zig!

Yeah, I think that's the assumption people had been operating under.

That team would have thoroughly endorsed a zig rewrite! It was a very odd situation where most of us liked erlang the language but found the beam to be an annoying beast, whereas most of the world seems to be the opposite.

Re: Elixir saves Pinterest $2M a year in server costs

#392

Post by hu3 »

moomoo11 wrote: post #292

I use Go and it gives me a good sweet spot between cost and perf. I’m nowhere close to performance requirements of Pinterest. But my auth service runs on a single 1vcpu, 512mb ram, and 10gb ssd. I use leveldb and swap on the 10gb. I’ve benchmarked it to handle 8-9k rps while delivering 150ms max response time. Not bad for a few bucks a month.

Good thing about Go is that it is easy to learn and hire for.

Re: Elixir saves Pinterest $2M a year in server costs

#393

Post by weatherlight »

jatins wrote: post #157
kramerger wrote: post #111

> José: Why did you choose Elixir? > Steve: We chose Elixir because we were looking for a system that was easy for programmers to understand and could take better advantage of our servers. I was intrigued at Elixir’s combination of friendly syntax, powerful metaprogramming features, and incorporation of the Actor model. Am I wrong or is this guy in some sort of bubble where only functional languages are taught?

I like Elixir as much as any other guy but I am not going to chose it for my team because it was "easy for programmers to understand" If you wanted something that was easy you'd go got Golang or Java, not a language without types. Lots of reasons to love Elixir but this doesn't sound like one

I think dynamic langs are easier to pick up than statically typed ones. But thats just me, I guess.

Re: Elixir saves Pinterest $2M a year in server costs

#394

Post by weatherlight »

paxys wrote: post #194
impulser_ wrote: post #58

Earlier quoted context omitted.

It more than just server costs. They reduced the complexity of their services by a lot. Having to maintain 95% less servers is worth it even if they didn't save any money IMO. This also could lead to them reducing their engineering team that maintains these services which would reduce costs even more.

Can switching to an esoteric functional language from Python and Java really be considered reducing complexity? No matter how well it is written, I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it.

> Can switching to an esoteric functional language from Python and Java really be considered reducing complexity?

It's not esoteric. It's a modern programing language, with 35 years behind its Virtual machine.

1 in 3 phone calls you've ever made was routed through a telephon switch written in a lang implemented in BEAM virtual machine.

WhatsApp was written in a lang implemented on the BEAM.

> I'm willing to bet that way fewer people in the company/industry understand the new codebase and can make changes to it.

It's really not hard to learn. It's not Haskell, It's not Coq, It's not Brainfuck.

You could pick up the syntax and the major semantics in a weekend.

Re: Elixir saves Pinterest $2M a year in server costs

#395

Post by weatherlight »

bluesnowmonkey wrote: post #371
brigadier132 wrote: post #348

Earlier quoted context omitted.

I was there and it was not being used in new projects. Maybe that spam filtering service has existed for that long. I would definitely not say that pinterest has fully bought into using elixir because of cost savings which is what the article seems to be insinuating.

There was a point like 8 (ish?) years ago where Pinterest wanted to consolidate on a single backend language internally, after it became apparent Python was not suitable, and multiple contenders were being tried in different places. I think at that time Steve was advocating for Elixir to be it. However it ended up being Java. Nowadays, kind of ironically after the consolidation, we have backend services in at least s…

that's unfortunate and makes me sad.

Re: Elixir saves Pinterest $2M a year in server costs

#396

Post by DoesntMatter22 »

fidotron wrote: post #254
pdimitar wrote: post #247

Earlier quoted context omitted.

Frankly I am seeing that as a myth, you seem to have made up your mind some time ago or judged by 1-2 occasions. I am on ElixirForum every day and worked with Elixir for 7 years and have never seen anyone "perpetuate myths". I've seen some people willing to "increase adoption" which was always met with resistance by the wider community -- we believe growth should be organic. Pretty sad stance from you though, I have…

> If you are not willing to try it, that's fair. Say that. Claiming you know stuff about the ecosystem while a guy who is there every day comes across as... strange. Biased. And not arguing in good faith. :( I am not willing to drag others, such as those that wrote the repos, into a technical discussion with people out to act as you are.

The guy you are responding to was completely calm and reasonable. Didn't say anything attacking or otherwise. I'm not sure why you are seemingly trying to cast him (and the Beam) in such a bad light, with seemingly no reason to back it up.

Re: Elixir saves Pinterest $2M a year in server costs

#397

Post by weatherlight »

sgt wrote: post #287

Clojure comes to mind. I think personally I'd rather rewrite into Clojure than Elixir. If only for the sake of REPL.

Elixir has a repl too! and I mean a repl in the LISP sense of a REPL. I can open a repl on my machine and peer into a production machine and its internals.

Re: Elixir saves Pinterest $2M a year in server costs

#398

Post by weatherlight »

cutler wrote: post #277

Whatever merits of Elixir there are only 10 Elixir jobs in the whole of the UK according to Indeed.com. Adopiton matters.

there's a joke that there reason there's so few jobs is 1) they are very high paying, and that the engineers are so productive they only need a few engineers to get work done.

Re: Elixir saves Pinterest $2M a year in server costs

#399

Post by winter_blue »

DoesntMatter22 wrote: post #375
winter_blue wrote: post #279

Earlier quoted context omitted.

> Do dynamically-typed languages inherently damage the environment? Or is it the fossil fuels? My opinion is that slow languages that use 10x the electricity, with no ROI for the 10x energy use is bad. High energy use, even if it's clean energy, implies a higher environmental toll. If a country were solely using nuclear and solar, higher energy use results in (1) more nuclear reactors constructed, and (2) more solar…

For the vast majority of projects this makes no difference. If you are at Facebook scale? Sure. But then, you do what they did, write a VM to speed things up. Dynamic typing doesn't cost much more money on average and thankfully the cost of energy itself is a motivation for companies to do rewrites. If they are paying a lot in server costs and electricity then they do typically rewrite to reduce the amount of servers…

> running at a profit and getting to market quickly which dynamic languages do extremely well

This (or similar variants of this) is an assertion that's commonly made about dynamically-typed languages, but I don't think they hold any water.

Less readable code (due to the lack of types) makes it a lot harder to add new features, and harder to debug code as well.

Several years ago, I briefly worked on a fairly large codebase at a startup that was written in a Ruby on the backend, and CoffeeScript on the front-end. There were only around 60,000 active users. Yet, the dynamic typing made adding new features, or fixing a bug a truly painful and fragile experience. Needlessly painful, and slow. It literally reduced developer velocity.

I think once you cross a few hundred lines, dynamic typing becomes a handicap rather than an advantage.

All of this doesn't even touch on the energy use. Which I'll admit is irrelevant to most companies. Server costs even for popular web/SaaS/etc tech companies are often a tiny tiny fraction of overall cost, with most of the company's annual operating cost being employee salaries. (As I had stated earlier, I don't mind a language being slower - if it actually provided any advantages–like improved developer productivity, in exchange for that slowness.)

Post reply on HN