Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

221–230 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#221

Post by throwaway894345 »

pdimitar wrote: post #196
miroljub wrote: post #181

Earlier quoted context omitted.

Even if we take this as a factor, there's still no reason why would a Go or Java or SBCL implementation be significantly worse than Elixir.

There is a reason: the BEAM is almost not prone to huge GC pauses. Bigger load results in every actor responding very slightly slower. Nothing else. Many other systems don't have this property. They fall down under pressure. Gosh, a huge chunk of HN is always so dismissive. At least read up a bit beforehand, man. The criticisms should be informed and benefit the readers, not only express a generic skepticism.

Go also doesn’t have huge GC pauses (and moreover idiomatic Go generates very little garbage)and I have a hard time seeing how GC pauses would contribute so significantly. Java also allegedly has a very low latency collector.

Re: Elixir saves Pinterest $2M a year in server costs

#222

Post by dahwolf »

morelisp wrote: post #112
coldtea wrote: post #93

Earlier quoted context omitted.

> Elixir is slower than plain PHP according to the techempower benchmarks. I'm not even sure how that's possible but it is. You're "Not even sure how that's possible" as if implying PHP is slow? Even old pre-7 PHP was much faster than Ruby, Python, and others. And most PHP libs are just wrappers over C code.

> Even old pre-7 PHP was much faster than Ruby, Python, and others. No, PHP 7 was an impressive step forward in speed because it switched to better bytecode and object representations internally, but Python wiped the floor with most versions of PHP 5 for exactly the same reason. (PHP 5.5 with opcaching was roughly comparable.) But this comparison overall is like trying to find the strongest two-year-old...

It's a ridiculous thing to say that any version of PHP is slow. You could run PHP5 today and it's still not slow by any modern standard. Even on the hardware of 15 years ago almost any script you throw at it finishes in single digit milliseconds.

How is that slow? I'd like to meet the developers that consider this slow.

Re: Elixir saves Pinterest $2M a year in server costs

#223

Post by brd529 »

amarshall wrote: post #216
winter_blue wrote: post #191

Earlier quoted context omitted.

This is a pretty realistic balanced real-world benchmark of web frameworks (and languages): https://www.techempower.com/benchmarks/ It shows that Python with Django is literally 40 times slower than the fastest framework. Python with uvicorn is 10 times slower. The use of languages like Python and Ruby literally results in >10x the servers being used; which not only results in higher cost, but also greater electricit…

Sure, but the fastest Elixir-based entry on that list has half the score of uvicorn (Python), so not really sure what your point is here w.r.t. OP.

Reading higher in the thread, some Elixir folks are saying that the techempower benchmarks used the wrong settings (debug mode, etc) for their Elixir benchmark.

Re: Elixir saves Pinterest $2M a year in server costs

#224

Post by EMM_386 »

andrewstuart wrote: post #155

200 servers running Python. I bet it was an architecture problem more than a language problem. I’d be expecting to leave most of that in place and fix the hotspots that are dragging performance down. Maybe some compiled code in certain places. Almost certainly a detailed analysis would have yielded 20 things that could be tuned. Posts like this are almost something to be ashamed of “we rewrote an entire subsystem bec…

> Posts like this are almost something to be ashamed of “we rewrote an entire subsystem because we wanted to use a language we like”. That’s really failing in your responsibility to the company.

I would tend to agree. I don't understand these sorts of decisions to rewrite huge sites in niche languages and frameworks. You tend to get one engineer who really likes some language, has some clout, and is able to pitch it as a good idea. And then the blog posts about how it saves the company untold millions, while hiding all of the associated costs.

Not only is it niche, so your costs going forward will be higher, it even required investing in new "tools and strategies" just to bechmark it.

"Elixir has proven so efficient that testing the limits of our services became a challenge unto itself, requiring investment in new benchmarking tools and strategies".

I wouldn't personally gamble on a robust job market for Elixir developers in a decade. And those there are will be able to command higher wages.

Sure, they will be out there. But so were COBOL developers for Y2K.

Re: Elixir saves Pinterest $2M a year in server costs

#225

Post by pdimitar »

SanderNL wrote: post #205
pdimitar wrote: post #196

Earlier quoted context omitted.

There is a reason: the BEAM is almost not prone to huge GC pauses. Bigger load results in every actor responding very slightly slower. Nothing else. Many other systems don't have this property. They fall down under pressure. Gosh, a huge chunk of HN is always so dismissive. At least read up a bit beforehand, man. The criticisms should be informed and benefit the readers, not only express a generic skepticism.

That does not in any way explain a drop of 95%, which IMO is ridiculous and points to other issues. The system they created now is totally different from the one they had. It’s more efficient by an insane margin. Choice of language seems like it would have trouble breaking the top 5 major reasons.

Why is it ridiculous?

I migrated Rails apps to Elixir before, we reduced from 15 servers to 3, and 1 was basically "if crap hits the fan", we could have gotten away with 2 easily.

It's worrying that a supposedly high-quality forum like HN receives comments with no substance. If you have an actual counter-argument, let's discuss. If not, well, not an interesting exchange.

Re: Elixir saves Pinterest $2M a year in server costs

#226

Post by wredue »

pdimitar wrote: post #197
jeswin wrote: post #151

Earlier quoted context omitted.

> ... with 1/50th the effort of any other language. To junior developers reading this, such statements are never true.

What's your counter-argument?

They don’t need one.

When a person makes a claim, especially such a ridiculous one, it is perfectly valid to outright reject that claim without any argument. Why? Because no argument was provided in the first place.

Re: Elixir saves Pinterest $2M a year in server costs

#227

Post by throwaway894345 »

83457 wrote: post #188
miroljub wrote: post #181

Earlier quoted context omitted.

Even if we take this as a factor, there's still no reason why would a Go or Java or SBCL implementation be significantly worse than Elixir.

Isn't the major difference here compared to others the BEAM?

You need to advance a theory about how the BEAM could make a 95% improvement in a way the alternatives cannot.

For example, a Go has a very good asynchronous story as well and while it may not be exactly as good as BEAM languages, it makes up for it in straight line execution performance.

I’ve personally rewritten a few carefully optimized Python applications in pretty naive Go (without significant rearchitecting) and witnessed 50X-1000X performance improvements. And moreover Go allows for more optimization beyond what Python allows (e.g., consolidating allocations and lifting them out of the hot path).

Re: Elixir saves Pinterest $2M a year in server costs

#228

Post by pdimitar »

fidotron wrote: post #210
pdimitar wrote: post #196

Earlier quoted context omitted.

There is a reason: the BEAM is almost not prone to huge GC pauses. Bigger load results in every actor responding very slightly slower. Nothing else. Many other systems don't have this property. They fall down under pressure. Gosh, a huge chunk of HN is always so dismissive. At least read up a bit beforehand, man. The criticisms should be informed and benefit the readers, not only express a generic skepticism.

The beam just garbage collects each actor separately, and it so happens that much of the time your actor has finished before a gc happened so you never see the cleanup. The beam also has a spectacular failure mode: OOM whenever messages come in at a higher rate than they are processed. The lack of backpressure mechanisms mean a huge amount of beam language developers spend way too much time recreating their own way f…

I can see how you would think that, yes. In practice I haven't noticed it except in super rare cases where processes (actors) hold on to huge binaries / strings -- which is one of the weak points of BEAM's GC.

Re: Elixir saves Pinterest $2M a year in server costs

#229

Post by fidotron »

pdimitar wrote: post #228
fidotron wrote: post #210

Earlier quoted context omitted.

The beam just garbage collects each actor separately, and it so happens that much of the time your actor has finished before a gc happened so you never see the cleanup. The beam also has a spectacular failure mode: OOM whenever messages come in at a higher rate than they are processed. The lack of backpressure mechanisms mean a huge amount of beam language developers spend way too much time recreating their own way f…

I can see how you would think that, yes. In practice I haven't noticed it except in super rare cases where processes (actors) hold on to huge binaries / strings -- which is one of the weak points of BEAM's GC.

My experience of this is not theoretical.

Re: Elixir saves Pinterest $2M a year in server costs

#230

Post by pdimitar »

wredue wrote: post #226
pdimitar wrote: post #197

Earlier quoted context omitted.

What's your counter-argument?

They don’t need one. When a person makes a claim, especially such a ridiculous one, it is perfectly valid to outright reject that claim without any argument. Why? Because no argument was provided in the first place.

Well, I am trying to be better than them.

Also I dislike FUD spreading so there's that, I am trying to engage them in a more-informed manner.

Though I am not sure whom you are addressing, my parent poster or theirs.

Post reply on HN