Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Elixir saves Pinterest $2M a year in server costs

paraxial.io

Post reply on HN
Print view View original

211–220 of 481 posts

Re: Elixir saves Pinterest $2M a year in server costs

#211

Post by swarnie »

bradleykingz wrote: post #169
giancarlostoro wrote: post #133

Earlier quoted context omitted.

Maybe to you, but my wife uses Pinterest a ton. It is where a lot of the women I know go to for ideas for just about everything, from house decor to even the ideas for our wedding. Pinterest is useful just not for us nerdy guys. I am not sure why Google keeps it though or how they benefit, unless Pinterest uses AdSense exclusively, then one can determine that its some sort of partnership. You would think Google would…

It's one thing to use Pinterest intentionally, a whole other to have it shoved down your throat.

Where / How are you finding Pinterest links?

I've just searched about two years of browser history and i haven't landed their once.

Re: Elixir saves Pinterest $2M a year in server costs

#212

Post by liveoneggs »

miroljub wrote: post #171
coldtea wrote: post #3

By the title alone, this means nothing. How much would it cost otherwise? What is the percentage savings? In TFA, it gets better though: "Steve: That’s pretty easy. When I started on the spam team, we had close to 1,400 servers running. When we converted several parts to Elixir, we reduced that by around 95%. One of the systems that ran on 200 Python servers now runs on four Elixir servers (it can actually run on two…

Second system syndrome. They did savings by re-implementing their services and attribute those savings to the new tool / programming language. I wonder what the saving would look like if they chose another tool for the second / optimized system. I doubt it would differ much if they went with Go, Java or stayed with Python.

While you are probably right it's also true that python is just straight up slow - especially in normal configurations (django, flask, uwsgi, lambdas, etc) and elixir is pretty fast while offering a great/fun/friendly dev experience + BEAM-HA/scaling-benefits.

They could have also blown everything out of the water with C++ - or probably even golang - but if elixir can do it on 2-4 boxes it's fast enough.

Re: Elixir saves Pinterest $2M a year in server costs

#213

Post by varispeed »

Incolotopo wrote: post #165
varispeed wrote: post #143

Surely this means they could donate at least $1M a year to the Elixir team?

yes but besides a somehow snarky comment, it doesn't make sense from a company perspective. This wasn't done to have mone to donate to Elixir but for revenue. And besides that, you don't even know if they do

Big corporations have a habit of promoting open source, then gullible developers produce exceptional work from which said big corporations make millions or billions without paying a penny back.

Then developers learn they can't pay bills with exposure tokens and employment doesn't offer the lifestyle they had hoped for.

But there is no worry, they get replaced by next cohort of gullibles believing in open source.

What I am trying to say, it should be illegal for big corporations to use open source without paying royalties to all contributors.

Re: Elixir saves Pinterest $2M a year in server costs

#214

Post by 0xedd »

TacticalCoder wrote: post #23
threeseed wrote: post #17

Earlier quoted context omitted.

> When our notifications system was running on Java, it was on 30 c32.xl instances. When we switched over to Elixir, we could run on 15. Would be curious to know how they tried to optimise the Java stack. Because on every benchmark I've seen the JVM is faster in every which way than Elixir. Except for memory where often people will over-provision the JVM rather than look at where their code might be over-allocating o…

They rewrote, which is known to help too. Going from 30 to 15 instances is not bad but it's very likely that a Java-to-Java rewrite would have helped go down too. The big one however is going from 200 Python servers to 4 Erlang ones: a 50x reduction is quite something and a Python-to-Python rewrite would not have allowed to achieved a 50x gain: > All this is possible because Elixir, and the Erlang platform underneath…

They threw DB[s] and Kafka into the mix. Python would get them the same net gain, if not more, with less dev cost. Python's I/O workloads perform on par with Go/NodeJS (See FastAPIs 3rd party quarterly benchmarks as an example).

If they rebrand Python to Metal or some other name, people would recommend it left and right. It's just suffering from bandwagon criticism. Yet it remains one of the top 3 languages for years, covering several domains.

Re: Elixir saves Pinterest $2M a year in server costs

#215

Post by bearjaws »

giancarlostoro wrote: post #133
kramerger wrote: post #113

Earlier quoted context omitted.

That perfectly describes my usage of Pinterest. People have written browser extensions to remove Pinterest from search results, as it is almost always a dead-end

Maybe to you, but my wife uses Pinterest a ton. It is where a lot of the women I know go to for ideas for just about everything, from house decor to even the ideas for our wedding. Pinterest is useful just not for us nerdy guys. I am not sure why Google keeps it though or how they benefit, unless Pinterest uses AdSense exclusively, then one can determine that its some sort of partnership. You would think Google would…

They are referring to how they link out their images and SEO content that ultimately one will find unusable & frustrating.

I have experienced this many times when looking for a specific image on Google Images, I will click on the link that goes to Pinterest only to find the image is not there.

Re: Elixir saves Pinterest $2M a year in server costs

#216

Post by amarshall »

winter_blue wrote: post #191
miroljub wrote: post #171

Earlier quoted context omitted.

Second system syndrome. They did savings by re-implementing their services and attribute those savings to the new tool / programming language. I wonder what the saving would look like if they chose another tool for the second / optimized system. I doubt it would differ much if they went with Go, Java or stayed with Python.

This is a pretty realistic balanced real-world benchmark of web frameworks (and languages): https://www.techempower.com/benchmarks/ It shows that Python with Django is literally 40 times slower than the fastest framework. Python with uvicorn is 10 times slower. The use of languages like Python and Ruby literally results in >10x the servers being used; which not only results in higher cost, but also greater electricit…

Sure, but the fastest Elixir-based entry on that list has half the score of uvicorn (Python), so not really sure what your point is here w.r.t. OP.

Re: Elixir saves Pinterest $2M a year in server costs

#217

Post by Dowwie »

paxys wrote: post #187

So the cost of 3-4 FTEs at a company that employs >4000. Cool. And how much does the change cost them in terms of: - Not being able to hire talent when they need because they don't know or aren't willing to work with Elixir. - Spending extra time training engineers. - Having to hire more senior engineers/those versed in functional programming vs generalists. - The rewrite itself (I'm sure it didn't happen magically o…

It wasn't too long ago that engineers had to walk over hot coals to get their company to use Python. All of the things you're saying were applicable to Python when the company "only" had programmers who knew Java.

Re: Elixir saves Pinterest $2M a year in server costs

#218

Post by skrebbel »

arrowsmith wrote: post #203
capableweb wrote: post #192

Earlier quoted context omitted.

Monads and Haskell is not a requirement to understand a functional programming language. Also, I find it hard to believe that anyone who knows N+1 programming languages would have a hard time understanding Elixir quickly, it looks like most mainstream programming languages used today, with slightly different syntax for some things. Take a look at https://elixir-lang.org/crash-course.html and you'll see what I mean, i…

As much as I love Elixir, it's really not just "basically Ruby with some slight modifications". The syntax is similar at a glance, but there are some major design differences (e.g. immutability); solving the same problem in Elixir and Ruby can require a completely different structure to your code. I wrote this blog post about what I think are the three most important differences between Elixir and Ruby: https://phoen…

No monads though. No pure functions, no Option types, pretty much no recursion in practice. Really it’s got none of the stuff that makes some FP languages hard to learn. (Except maybe “no for loops”).

Re: Elixir saves Pinterest $2M a year in server costs

#219

Post by bluedino »

edanm wrote: post #179
paulsutter wrote: post #177

Earlier quoted context omitted.

Python is 100x slower than a real language (I like python and use it often, not meant as a dig at the language just stating facts) My favorite thing about Python is writing prototypes. The big risk with writing a prototype is that it survives into production. Using Python ensures that the code will be replaced by real code Python is great for other non-production code like Jupyter notebooks, numpy experiments, etc

How is calling Python "not a real language" just stating facts? I'm not sure by which metric we are judging whether a language is real or not, but I'm fairly sure that almost anything anyone comes up with will include Python, considering it's one of the most used languages in the world at this point, and used for a fairly large variety of use cases.

That 'fact' was that Python is slower than ____, not "Python isn't a real language"

Re: Elixir saves Pinterest $2M a year in server costs

#220

Post by edanm »

bluedino wrote: post #219
edanm wrote: post #179

Earlier quoted context omitted.

How is calling Python "not a real language" just stating facts? I'm not sure by which metric we are judging whether a language is real or not, but I'm fairly sure that almost anything anyone comes up with will include Python, considering it's one of the most used languages in the world at this point, and used for a fairly large variety of use cases.

That 'fact' was that Python is slower than ____, not "Python isn't a real language"

Oh ok I may have misunderstood that part of the comment (and considering it's so short, I guess that's a silly mistake on my part).

My comment still stands on saying that Python isn't a real language being a pretty non-sensical statement.

Post reply on HN