Live data from Hacker News

How And Why We Switched from Erlang to Python (2011)

engineering.mixpanel.com

111–120 of 150 posts

Re: How And Why We Switched from Erlang to Python (2011)

#111

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

If you have some developer that wants to write something in a new (for your company) language, or even worse goes off and does so without asking, because he wants to try it out, you probably want to fire that guy. Either he doesn't understand the downsides or he does, but doesn't care.

What if it has the potential to be a much better tool for the job than the mainstream options, and your developer thinks it's worth trying it to find out?

I've seen this situation with adopting a relatively unusual language for a task that needed particularly high reliability. Yes, it was a concern that only one person currently working in the organisation knew much about it and that it might be harder to hire help if more developers were required later. On the other hand, that software was developed faster than the similar system it was due to supersede, and it had a much lower rate of reported bugs once deployed.

Of course you don't want to hire people who intend to pad their resumes with extra buzzwords on your dollar when it has no business benefit, but equally, if you never use tools that aren't chosen from among a small set of mainstream options, you'll never outperform what you can do with mainstream tools.

Re: How And Why We Switched from Erlang to Python (2011)

#112

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

He should get fired for saying "concurrency story".

Re: How And Why We Switched from Erlang to Python (2011)

#113
The term « erlang expert » made me wonder : since erlang is supposed to be a much saner and stricter architecture ( share nothing, code inside the actor in a synchronous manner, etc), what are the usual troubles someone encounters performance-wise in that language ?

Apart from a particular cpu intensive task that would make just a single request perform badly, it seems ( from just reading about the language) that just following the guidelines ensures top performance.

Re: How And Why We Switched from Erlang to Python (2011)

#114

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

If you have some developer that wants to write something in a new (for your company) language, or even worse goes off and does so without asking, because he wants to try it out, you probably want to fire that guy. Either he doesn't understand the downsides or he does, but doesn't care. What if it has the potential to be a much better tool for the job than the mainstream options, and your developer thinks it's worth t…

Software development is much more than just programming languages, I have seen teams outperform with lousy languages thanks to a great team spirit and understanding of customer needs.

Re: How And Why We Switched from Erlang to Python (2011)

#115
post #108

Earlier quoted context omitted.

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

If there are no other people who can maintain that stack, maybe (firing is too extreme, I’d just ban the Clojure code in the production) Writing initial version of the code is only a small part of it. Maintaining and enhancing it is much harder. What is the company going to do if the new endpoint went down and the author is on vacation?

A lot of these objections depend on context, though. The culture at large organisations taking on large and/or long-running projects is necessarily different to the culture at small organisations that can't afford the same overheads and wouldn't enjoy the advantages of greater redundancy or interchangeability anyway.

Re: How And Why We Switched from Erlang to Python (2011)

#116
post #114

Earlier quoted context omitted.

If you have some developer that wants to write something in a new (for your company) language, or even worse goes off and does so without asking, because he wants to try it out, you probably want to fire that guy. Either he doesn't understand the downsides or he does, but doesn't care. What if it has the potential to be a much better tool for the job than the mainstream options, and your developer thinks it's worth t…

Software development is much more than just programming languages, I have seen teams outperform with lousy languages thanks to a great team spirit and understanding of customer needs.

For sure. A good craftsman will do good work with any reasonable set of tools.

They'll still achieve their best possible results and performance with high quality tools of their own choosing, though.

Re: How And Why We Switched from Erlang to Python (2011)

#117

Earlier quoted context omitted.

Well, Rails does not have a good track record with scaling. Memory requirements if nothing else will be a big hurdle to come by regardless if you have SOA, in my experience. Back in the day not many organizations survived on Ruby/Rails and many of them moved over to Erlang or Java just to deal with resource issues. https://blog.chef.io/2013/02/15/the-making-of-erchef-the-che... Not sure what is up with Ruby nowadays.…

I'm not an authoritative source by any stretch but my anecdotal evidence shows many former colleagues getting a Rails fatigue of sorts and moving to other languages. I personally moved on to Elixir but no small amount went back to Python and PHP and some got to JS, Haskell, even OCaml and Rust and Go. It's really not productive when even minor version bumps of Rails back in the days of v3.2 all the way to v4.2 were a…

Yeah I have seen exactly the same.

Re: How And Why We Switched from Erlang to Python (2011)

#118

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Why would it be madness to us javascript on the backend?

Re: How And Why We Switched from Erlang to Python (2011)

#119

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

Another developer: "...our stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. I just bypassed the ORM and wrote a slightly more complex SQL query, so that there is no more N+1 query problem in the backend. The latency of this end point was cut by several orders of magnitude, and our database is not overloaded by parallel inefficient queries, run by clients with 'great concurrency stories'."

Re: How And Why We Switched from Erlang to Python (2011)

#120
post #110
post #94

Elixir, which runs on the Beam VM, had piqued my interest for a while, so finally set out to learn about it this weekend. On searching for "flaws" of the language and the platform, I (accidentally) ran into: https://www.youtube.com/watch?v=42k70Y-yTYY . (year 2017) Video description: ...What many developers don’t understand is that Erlang is built on an architecture and within ecosystem that contains many subtle secu…

This is an issue that’s overblown, simply have a firewall that exposes used ports and you’re fine. There’s also an option within the release tool distillery to limit it to the local network. Not to mention even if an attacker found your unsecured setup, they’d also need to know your “cookie”/key to do anything. It’s no different than leaving SSH login with password enabled on any server.

The speaker claims that the cookie/key can be known by causing VM crash, which is done by exploiting the monotonic nature of cookie values which are stored as atoms which has a fixed cap on how many they can be, which when exceeded causes VM to crash. He estimates the time to do that be 10 mins (I forgot the exact memory limit)
Post reply on HN