Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How And Why We Switched from Erlang to Python (2011)

engineering.mixpanel.com

Post reply on HN
Print view View original

21–30 of 150 posts

Re: How And Why We Switched from Erlang to Python (2011)

#21

Post by rrix2 »

tombert wrote: post #18

In some fairness to the writer, Erlang has improved a lot since 2011. I'm a big fan of the Erlang platform, and I even kind of begrudgingly love the language, but Erlang didn't even have first-class immutable maps until 2015, and while I actually think the semi-prolog pattern-matching syntax is elegant and beautiful once you get a handle on it, I actually really dislike the whole comma-semicolon-period thing. (Person…

Are you writing LFE in your professional capacity? I've always been curious if LFE was more than a sort of toy language, it's hard to tell from outside the community. I've been becoming more and more enticed to learn Elixir (and Phoenix) and the BEAM, and LFE would be a nice thing to add to the tool belt.

Re: How And Why We Switched from Erlang to Python (2011)

#22

Post by vector_spaces »

rrix2 wrote: post #21
tombert wrote: post #18

In some fairness to the writer, Erlang has improved a lot since 2011. I'm a big fan of the Erlang platform, and I even kind of begrudgingly love the language, but Erlang didn't even have first-class immutable maps until 2015, and while I actually think the semi-prolog pattern-matching syntax is elegant and beautiful once you get a handle on it, I actually really dislike the whole comma-semicolon-period thing. (Person…

Are you writing LFE in your professional capacity? I've always been curious if LFE was more than a sort of toy language, it's hard to tell from outside the community. I've been becoming more and more enticed to learn Elixir (and Phoenix) and the BEAM, and LFE would be a nice thing to add to the tool belt.

It looks like the last commit to LFE was about a year ago (not that this necessarily makes it a toy or not)

https://github.com/rvirding/lfe/branches

Re: How And Why We Switched from Erlang to Python (2011)

#23

Post by robertAngst »

vidarh wrote: post #17
pdimitar wrote: post #8

Them not being able to utilize Erlang well seems to be the gist of the article. The rest seems like a post-hoc rationalization. It's fair to switch away from a technology your team doesn't know well and doesn't want to learn. Maybe that should have been the title.

Long time, Dimi.. It's a favorite pet peeve of mine too that people often try to rationalize either their lack of experience with a platform or a broken architecture as a problem with the language in question. Though sometimes it's done because it helps paper over internal politics (e.g. lack of clout to be perceived to criticize past decisions) by blaming something external like a language or framework to justify ar…

> they'd written a monolith instead of building a system where the message delivery was designed to be sharded; n

Do some languages handle this differently?

Isnt this based on how the code was written rather than a language natively sharding?

Re: How And Why We Switched from Erlang to Python (2011)

#24

Post by vikeri »

rrix2 wrote: post #21
tombert wrote: post #18

In some fairness to the writer, Erlang has improved a lot since 2011. I'm a big fan of the Erlang platform, and I even kind of begrudgingly love the language, but Erlang didn't even have first-class immutable maps until 2015, and while I actually think the semi-prolog pattern-matching syntax is elegant and beautiful once you get a handle on it, I actually really dislike the whole comma-semicolon-period thing. (Person…

Are you writing LFE in your professional capacity? I've always been curious if LFE was more than a sort of toy language, it's hard to tell from outside the community. I've been becoming more and more enticed to learn Elixir (and Phoenix) and the BEAM, and LFE would be a nice thing to add to the tool belt.

If you like the BEAM and Lisp, check out Clojerl too: https://github.com/clojerl/clojerl by Juan Facorro

Re: How And Why We Switched from Erlang to Python (2011)

#25

Post by giancarlostoro »

dragonwriter wrote: post #16
giancarlostoro wrote: post #10

Sometimes using the right tool for the right job also means making sure you have people who can use said tool.

Or, part of the job that the tool has to be right for is the team, which can be as much a part of the job definition as the output. OTOH, if there is strong reason to think the tool is otherwise correct, finding resources to. enable the team to gain and/or borrow the knowledge they are currently lacking should be practical.

> OTOH, if there is strong reason to think the tool is otherwise correct, finding resources to. enable the team to gain and/or borrow the knowledge they are currently lacking should be practical.

It's incredible companies don't invest in training enough, and this goes both for the companies who provide training (causing companies to avoid wasting money on it to begin with) and the ones who need training for their teams.

A senior I worked with mentioned how he learned OO when his company he worked at had everybody trained on it, and hasn't forgotten it since.

But now it's all the job of the degrees right. They'll totally cover everything your company needs to use every obscure tool you want to be used for pennies on the dollar.

Re: How And Why We Switched from Erlang to Python (2011)

#26

Post by robertAngst »

phoe-krk wrote: post #12

> No one on our team is an Erlang expert, and we have had trouble debugging downtime and performance problems. So, we decided to rewrite it in Python, the de-facto language at Mixpanel. So they have changed the language to suit the programmers instead of changing programmers to suit the language.

>Pick a language that has limited programmers, bring programmers from around the country, pay them well beyond 6 figures so they don't leave and have a crisis every time a programmer leaves.

OR

>Go with a mainstream language that they teach every CS kid and has every googable question imaginable in SO, hire 1 experienced programmer to manage a bunch of post-college-kids.

There are lots of good reasons to pick a language with a strong community. Once in a while, I hear people recommend obscure languages(or up-and-coming), and I think- that is going to be expensive to maintain.

Re: How And Why We Switched from Erlang to Python (2011)

#27

Post by lightcatcher »

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebase. Eventually, it became painful to debug or add features to this endpoint because no one was particularly proficient in Erlang. Ankur (author of the post, intern) then rewrote it in Python.

I wouldn't read this as a negative about Erlang or a positive article about Python. I'd instead read it as "use a language that makes sense for your organization and already existing codebase".

Re: How And Why We Switched from Erlang to Python (2011)

#28

Post by jlouis »

nemothekid wrote: post #3

1. This article was written in 2011 (it's 8 years old). I was confused why they were going with eventlet, but the age of the article explains why. 2. Even for 2011, 1k rps with an average latency of 100ms seems laughably bad. I have to be missing something here.

To be fair, the text doesn't seem to say if we are looking at maximum or average latency. Depending on what is being measured, you will have vastly different opinions on what is good and/or bad latency.

Also, note ratio of clients to server is 100:1. So 100ms latency almost has to be the maximally experienced: that single core node cannot spend more than 1ms per request on average, or it will fall behind.

The classic problem with these benchmarks are that they fall to what Gil Tene calls coordinated omission. I can park some of the clients for a while and process the other clients far quicker. It improves the req/s and if I'm just coming back to the parked requests within, say, 100ms, everything will look fine. But there is a lot of variance on response times.

Re: How And Why We Switched from Erlang to Python (2011)

#29

Post by y4mi »

avip wrote: post #20
wqwh wrote: post #14

Earlier quoted context omitted.

When you write Python you already know you're throwing the performance out of the window. In fact based on my experience those numbers are pretty good.

It's trivial to get 1kr/s with the simplest of wsgi flask behind nginx without touching anything. But these "comparisons" are meaningless without asking what "serving the request" is about.

as much as i love the python ecosystem and django in particular, the parent is definitely correct.

Don't ever choose python for performance. Its definitely fast enough for most things, but its just not the tool to leverage if you actually need performance.

Its wonderful if you just want to get shit done though, there are just so many good libraries around that you can use to get your project finished without committing several times the effort it takes in other languages

Re: How And Why We Switched from Erlang to Python (2011)

#30

Post by kvakvs »

robertAngst wrote: post #26
phoe-krk wrote: post #12

> No one on our team is an Erlang expert, and we have had trouble debugging downtime and performance problems. So, we decided to rewrite it in Python, the de-facto language at Mixpanel. So they have changed the language to suit the programmers instead of changing programmers to suit the language.

>Pick a language that has limited programmers, bring programmers from around the country, pay them well beyond 6 figures so they don't leave and have a crisis every time a programmer leaves. OR >Go with a mainstream language that they teach every CS kid and has every googable question imaginable in SO, hire 1 experienced programmer to manage a bunch of post-college-kids. There are lots of good reasons to pick a langu…

You do not need a large team of Erlang developers though, a much smaller team can hold the project afloat, even hire consultants to fix it and leave it for a few years. Also Erlang dev salaries aren't that stellar, just average what you'd pay a Python or a Java person.
Post reply on HN