Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

181–190 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#181

Post by miroljub »

throwaway894345 wrote: post #175
miroljub wrote: post #171

Earlier quoted context omitted.

Second system syndrome. They did savings by re-implementing their services and attribute those savings to the new tool / programming language. I wonder what the saving would look like if they chose another tool for the second / optimized system. I doubt it would differ much if they went with Go, Java or stayed with Python.

I think second syndrome is probably a significant factor, but I can also believe that ditching Python was also a significant factor. EDIT: I’m being rate limited because I guess my comments are too spicy for the HN mods, but anyway I agree that there’s no reason other non-Python languages would fare much worse than Elixir.

Even if we take this as a factor, there's still no reason why would a Go or Java or SBCL implementation be significantly worse than Elixir.

Re: Elixir saves Pinterest $2M a year in server costs

#182

Post by pdpi »

mcintyre1994 wrote: post #106
_m8fo wrote: post #100

Earlier quoted context omitted.

[flagged]

I think Node is still single core by default? Elixir (or rather the Beam) will handle core utilisation so if you start a load of Elixir processes they’ll be spread across multiple cores.

> I think Node is still single core by default?

It’s complicated.

Node itself has never been single-threaded. The execution model for JavaScript is single-threaded, so there’s no working around that, but libuv uses threads to build async IO on top of blocking operations.

Then there’s worker threads, which are pretty similar to web workers AIUI, that give you parallel execution for cpu-intensive work.

Obviously, though, none of these facilities compare with BEAM

Re: Elixir saves Pinterest $2M a year in server costs

#184

Post by pclmulqdq »

andrewstuart wrote: post #155

200 servers running Python. I bet it was an architecture problem more than a language problem. I’d be expecting to leave most of that in place and fix the hotspots that are dragging performance down. Maybe some compiled code in certain places. Almost certainly a detailed analysis would have yielded 20 things that could be tuned. Posts like this are almost something to be ashamed of “we rewrote an entire subsystem bec…

And they got a 50x reduction in server cost - 200 servers down to 4. I have never heard of a performance optimization project with such substantial gains without a complete rewrite.

Re: Elixir saves Pinterest $2M a year in server costs

#186

Post by TeeWEE »

edanm wrote: post #179
paulsutter wrote: post #177

Earlier quoted context omitted.

Python is 100x slower than a real language (I like python and use it often, not meant as a dig at the language just stating facts) My favorite thing about Python is writing prototypes. The big risk with writing a prototype is that it survives into production. Using Python ensures that the code will be replaced by real code Python is great for other non-production code like Jupyter notebooks, numpy experiments, etc

How is calling Python "not a real language" just stating facts? I'm not sure by which metric we are judging whether a language is real or not, but I'm fairly sure that almost anything anyone comes up with will include Python, considering it's one of the most used languages in the world at this point, and used for a fairly large variety of use cases.

- GIL

- No sound typing

- No proper package management

- Runtime type checking only

- Interpreted

I guess thats what he means with "Not a real language".

I disagree.... Its a tool that has it place. But its often used as a hammer.

Re: Elixir saves Pinterest $2M a year in server costs

#187

Post by paxys »

So the cost of 3-4 FTEs at a company that employs >4000. Cool.

And how much does the change cost them in terms of:

- Not being able to hire talent when they need because they don't know or aren't willing to work with Elixir.

- Spending extra time training engineers.

- Having to hire more senior engineers/those versed in functional programming vs generalists.

- The rewrite itself (I'm sure it didn't happen magically overnight).

Unless the number had a few more zeros at the end of it, there's no scenario where this project made real business sense vs being some principal engineer's vanity project. $2M is what a company like Pinterest spends on an off-site or a random exec's spot bonus. There are probably analytics dashboards that no one looks at that cost >$2M/yr to produce.

Re: Elixir saves Pinterest $2M a year in server costs

#188

Post by 83457 »

miroljub wrote: post #181
throwaway894345 wrote: post #175

Earlier quoted context omitted.

I think second syndrome is probably a significant factor, but I can also believe that ditching Python was also a significant factor. EDIT: I’m being rate limited because I guess my comments are too spicy for the HN mods, but anyway I agree that there’s no reason other non-Python languages would fare much worse than Elixir.

Even if we take this as a factor, there's still no reason why would a Go or Java or SBCL implementation be significantly worse than Elixir.

Isn't the major difference here compared to others the BEAM?

Re: Elixir saves Pinterest $2M a year in server costs

#189

Post by adamauckland »

TekMol wrote: post #66

My problem with Elixir is that the syntax is too noisy: defmodule Math do def sum(a, b) do a+b end end Compare that with Python: def sum(a, b): return a+b Having the name of the file be the name of the module and whitespace defining the blocks is the perfect approach for me.

Yes, but they were moving from enterprise JAVA which is considerably more verbose

Re: Elixir saves Pinterest $2M a year in server costs

#190

Post by fsloth »

jeswin wrote: post #151
sph wrote: post #64

Earlier quoted context omitted.

The advantages of Elixir are not performance-related. There is a lot of focus on raw performance on web-related services, when in reality most of their running time is spent waiting for IO. If there are two things the BEAM excels at, is IO and turning almost any problem into half a dozen processes that are scheduled and run in parallel, if not geographically-distributed, with 1/50th the effort of any other language.…

> ... with 1/50th the effort of any other language. To junior developers reading this, such statements are never true.

It does not sound too incredible to me.

Erlang and BEAM was designed to handle Ericson's telephony services. Piping-lots-of-stuff-in-parallel in fault tolerant fashion is the main use case around which it was designed.

Of course this is anecdotal.

Post reply on HN