Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How And Why We Switched from Erlang to Python (2011)

engineering.mixpanel.com

Post reply on HN
Print view View original

61–70 of 150 posts

Re: How And Why We Switched from Erlang to Python (2011)

#61

Post by hu3 »

i0exception wrote: post #41

Disclaimer: I work at Mixpanel We have a monorepo and I imported our entire commit history into Mixpanel - https://twitter.com/i0exception/status/1010663994435067904 As you can see, over the last 3 years, we've rewritten large parts of our infrastructure in golang. While we still use python for a lot of things, we felt that the type safety and concurrency primitives in go were a much better fit for writing some of ou…

250k lines of Go in ~3 years, around 1k per weekday is impressive.

Adding 1k lines of code per day to a monorepo while keeping the project manageable is no easy feat.

Is it split into microservices? Any tips to tame the beast?

Re: How And Why We Switched from Erlang to Python (2011)

#62

Post by macspoofing »

erlangNewb wrote: post #42
lightcatcher wrote: post #27

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

Seems like an oversight that nobody thought to get proficient with erlang. As someone who has started learning it, this seems like the obvious choice.

>Seems like an oversight that nobody thought to get proficient with erlang.

If it's a component that nobody owns, and nobody cares about, then I'm not very surprised.

>As someone who has started learning it, this seems like the obvious choice.

The other obvious choice is to rewrite it.

Re: How And Why We Switched from Erlang to Python (2011)

#63

Post by macspoofing »

_RPM wrote: post #35
lightcatcher wrote: post #27

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

Is it common for an intern to get such an impactful project?

Depends on the intern, and the component. If the intern is talented, the component is well-tested and the mentor or supervisor can do an effective code-review - then why not?

Re: How And Why We Switched from Erlang to Python (2011)

#64

Post by michaelterryio »

bradleyjg wrote: post #54
lightcatcher wrote: post #27

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Erlang excels at being used in mixed-language shops. Erlang is an incredible control plane language.

Re: How And Why We Switched from Erlang to Python (2011)

#65

Post by nojvek »

bradleyjg wrote: post #54
lightcatcher wrote: post #27

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Language diversity is a pragmatic balance. Sometimes a language offers superior abstraction or some guarantees that eliminate a whole host of bugs and manual effort.

It’s always worth trying a hypothesis in a small spike and reliably proving whether it will scale to the rest of the org and the impact will be worth the effort.

But you’re right, there should be one official supported way of doing some kind of thing at a company.

Re: How And Why We Switched from Erlang to Python (2011)

#66

Post by reducesuffering »

i0exception wrote: post #41

Disclaimer: I work at Mixpanel We have a monorepo and I imported our entire commit history into Mixpanel - https://twitter.com/i0exception/status/1010663994435067904 As you can see, over the last 3 years, we've rewritten large parts of our infrastructure in golang. While we still use python for a lot of things, we felt that the type safety and concurrency primitives in go were a much better fit for writing some of ou…

Interesting stats. Am I interpreting it correctly that the average python dev is doing 6 commits a month at 25 lines per commit? 150 lines in 160 hours?

Re: How And Why We Switched from Erlang to Python (2011)

#67

Post by arendtio »

tdumitrescu wrote: post #39

FWIW the server in question is really simple. It has since been ported to Golang, which has become one of Mixpanel's primary languages (alongside Python and JS/TS).

Now this will make the Erlang people happy :D

Sometimes I have the impression that functional programmers dislike Go especially for having such an inelegant language design ;-)

Personally, I don't have hard feelings for either side: I love Go and like the functional style in general (I know a bit of Scheme). Erlang is one of my top 2 languages I want to learn (together with Rust).

Re: How And Why We Switched from Erlang to Python (2011)

#68

Post by giancarlostoro »

guitarbill wrote: post #47
giancarlostoro wrote: post #25

Earlier quoted context omitted.

> OTOH, if there is strong reason to think the tool is otherwise correct, finding resources to. enable the team to gain and/or borrow the knowledge they are currently lacking should be practical. It's incredible companies don't invest in training enough, and this goes both for the companies who provide training (causing companies to avoid wasting money on it to begin with) and the ones who need training for their tea…

A simpler explanation may be nobody wanted to learn Erlang, not that the company didn't want to provide training. I'm still not sure myself what the benefits of learning Erlang are with regards to my salary. I'm sure there are highly paid Erlang jobs out there, but so far, my career has never been defined by a single language.

For this particular company yes, I did say companies though speaking of the industry as a whole. It seems on the job training is no longer getting the type of investment it once did.

Re: How And Why We Switched from Erlang to Python (2011)

#69

Post by darkr »

reducesuffering wrote: post #58
ludwigvan wrote: post #52

Earlier quoted context omitted.

And I think I had read that Whatsapp used Erlang because Facebook Chat was using Erlang and they thought Facebook knew what they were doing and decided to use the same tech. What a twist

Where did you read that? I believe Whatsapp used erlang because they leveraged ejabberd [0] for XMPP. Wouldn't surprise me if this is why FB originally used Erlang for the chat servers too. [0] https://en.wikipedia.org/wiki/Ejabberd

Yep. Facebook chat was originally based on ejabberd.

Source: https://web.archive.org/web/20121224094022/http://cufp.galoi...

Re: How And Why We Switched from Erlang to Python (2011)

#70

Post by i0exception »

reducesuffering wrote: post #66
i0exception wrote: post #41

Disclaimer: I work at Mixpanel We have a monorepo and I imported our entire commit history into Mixpanel - https://twitter.com/i0exception/status/1010663994435067904 As you can see, over the last 3 years, we've rewritten large parts of our infrastructure in golang. While we still use python for a lot of things, we felt that the type safety and concurrency primitives in go were a much better fit for writing some of ou…

Interesting stats. Am I interpreting it correctly that the average python dev is doing 6 commits a month at 25 lines per commit? 150 lines in 160 hours?

Averages are a little misleading because pretty much everyone touches python only about half the engineering team uses it as their primary programming language. I don't have the median number of commits or lines per commit handy - but that would be a better approximation of developer productivity.
Post reply on HN