Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How And Why We Switched from Erlang to Python (2011)

engineering.mixpanel.com

Post reply on HN
Print view View original

101–110 of 150 posts

Re: How And Why We Switched from Erlang to Python (2011)

#101

Post by [deleted] »

vidarh wrote: post #17
pdimitar wrote: post #8

Them not being able to utilize Erlang well seems to be the gist of the article. The rest seems like a post-hoc rationalization. It's fair to switch away from a technology your team doesn't know well and doesn't want to learn. Maybe that should have been the title.

Long time, Dimi.. It's a favorite pet peeve of mine too that people often try to rationalize either their lack of experience with a platform or a broken architecture as a problem with the language in question. Though sometimes it's done because it helps paper over internal politics (e.g. lack of clout to be perceived to criticize past decisions) by blaming something external like a language or framework to justify ar…

[deleted]

Re: How And Why We Switched from Erlang to Python (2011)

#102

Post by scarface74 »

ajross wrote: post #57
exelius wrote: post #38

Earlier quoted context omitted.

Yeah; it’s actually pretty easy work. Interns have a lot of uncommitted time, and they’re going to leave soon so you don’t want to integrate them into the team too much. It’s also way easier to build to a well-defined spec (which an in-production endpoint would likely have). So you let them rewrite something that already works. If what they build is good, you‘ve just knocked out a good chunk of technical debt and fou…

The question isn't whether or not the work is easy for the intern, it's whether or not it's appropriate. And this cuts both ways: a company that is doing serious core refactoring/redevelopment on the back of an underpaid "intern" is probably exploiting that labor in an unfair way. If it's in fact an unpaid internship, then they're straight up in violation of the Fair Labor Standards Act. And, of course, a company tha…

As far as I know, unpaid internships have not really been a thing in software development.

I was an intern back in 1995 and they paid us $10/hour ($16.88 inflation adjusted) and they provided housing on a college dorm. To put that in perspective, the $650 I made after taxes every two weeks was the cost of tuition every quarter at the state school I attended. I made enough that summer to cover my entire senior year including books.

Re: How And Why We Switched from Erlang to Python (2011)

#103

Post by strmpnk »

dhab wrote: post #94

Elixir, which runs on the Beam VM, had piqued my interest for a while, so finally set out to learn about it this weekend. On searching for "flaws" of the language and the platform, I (accidentally) ran into: https://www.youtube.com/watch?v=42k70Y-yTYY . (year 2017) Video description: ...What many developers don’t understand is that Erlang is built on an architecture and within ecosystem that contains many subtle secu…

It seems to lack any context for the claims. It is true that the distributed Erlang protocol is not something you'd want to expose to other parts of your system with different privilege levels, but it's entirely possible to restrict what connects and how (if at all, it is an optional part of the system).

The main idea behind clustering is resilience of homogeneous components, not between varied applications or security domains. I do find it a bit disingenuous from that perspective. However the video is at least right on a few points:

* if you have control of one node, you can effectively control the connected nodes (if any) as well

* you can dynamically update and load code on all nodes (by design)

* you can connect to other nodes as a hidden node, though it's still technically visible, but you need to specifically look for it and it only is visible to the node you connect to (so be sure to monitor this on all nodes)

* the protocol itself is not designed to be public facing as timing attacks and cookie guessing are problems. (Cookies in this context are meant to prevent mistaken cluster connections and aren't really a security feature.)

* the clustering is a fully connected mesh so a malicious user could DOS a network by rapidly rebooting a node to clog existing communication and burn through ephemeral ports for connections to use (really only applies to very large clusters but it's good to know how well you can handle spurious node membership changes)

Having said that though, undetectable is a lazy claim (there are many ways to address this depending on how you setup your nodes and which epmd implementation you use) and the requirements are to keep the interface/ports open to things not part of the cluster. It's also far from a hard requirement to use the distribution feature at all if you don't trust your ability to setup an isolated network. It'd be the same issue if you had some external untrusted etcd or consul node connect to the rest and start screwing with consensus with similar devastation.

Lastly, there are other ways to cluster Erlang nodes which don't necessarily inherit disterl's problems (disterl = what is built-in but it can be replaced or augmented). Libraries like Partisan look very promising.

(EDIT: formatting)

Re: How And Why We Switched from Erlang to Python (2011)

#104

Post by bradleyjg »

tomjakubowski wrote: post #86
bradleyjg wrote: post #54

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

You could fire them or you could just help them understand something that they don't. Sheesh.

Fair. That part was over the top. But I do think employees pushing their own intellectual passions or resume building pursuits over the interests of their employer are pernicious.

If it’s out of ignorance, then sure by all means educate.

Re: How And Why We Switched from Erlang to Python (2011)

#105

Post by cellularmitosis »

bradleyjg wrote: post #54
lightcatcher wrote: post #27

I was an intern at Mixpanel at the time (but not the author of this article or involved in the discussed rewrite). For context, the company was then My recollection of events (of 8 years ago): The Erlang endpoint was written a year or two earlier as someone's first Erlang project, in a tiny org with no Erlang experience. The endpoint worked well enough and people moved onto other projects in the mostly Python codebas…

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency story and cut the latency by 75%."

lol yeah that guy should totally be fired.

Re: How And Why We Switched from Erlang to Python (2011)

#106

Post by ramchip »

dhab wrote: post #94

Elixir, which runs on the Beam VM, had piqued my interest for a while, so finally set out to learn about it this weekend. On searching for "flaws" of the language and the platform, I (accidentally) ran into: https://www.youtube.com/watch?v=42k70Y-yTYY . (year 2017) Video description: ...What many developers don’t understand is that Erlang is built on an architecture and within ecosystem that contains many subtle secu…

Erlang distribution is just not meant to be used over the Internet, both for security and network reliability reasons. It’s something you would use on a restricted LAN. It’s also entirely optional.

There’s an FAQ entry explaining the reasoning behind this - in short, making a distribution protocol that can work securely with untrusted nodes is very difficult and risky, so the more pragmatic choice is to just not support that use case.

https://github.com/erlang/otp/wiki/FAQ:-What-kind-of-patches...

Re: How And Why We Switched from Erlang to Python (2011)

#107

Post by fineoldcannibal »

darkr wrote: post #69
reducesuffering wrote: post #58

Earlier quoted context omitted.

Where did you read that? I believe Whatsapp used erlang because they leveraged ejabberd [0] for XMPP. Wouldn't surprise me if this is why FB originally used Erlang for the chat servers too. [0] https://en.wikipedia.org/wiki/Ejabberd

Yep. Facebook chat was originally based on ejabberd. Source: https://web.archive.org/web/20121224094022/http://cufp.galoi...

I believe it was because of the Beluga acquisition which was in Erlang

Re: How And Why We Switched from Erlang to Python (2011)

#108

Post by theamk »

cellularmitosis wrote: post #105
bradleyjg wrote: post #54

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

If there are no other people who can maintain that stack, maybe (firing is too extreme, I’d just ban the Clojure code in the production)

Writing initial version of the code is only a small part of it. Maintaining and enhancing it is much harder.

What is the company going to do if the new endpoint went down and the author is on vacation?

Re: How And Why We Switched from Erlang to Python (2011)

#109

Post by andy_ppp »

cellularmitosis wrote: post #105
bradleyjg wrote: post #54

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

The problem here is not understanding your database not about making a system in an unsupported and probably undeployable thing to prove how clever you are...

Re: How And Why We Switched from Erlang to Python (2011)

#110

Post by bdibs »

dhab wrote: post #94

Elixir, which runs on the Beam VM, had piqued my interest for a while, so finally set out to learn about it this weekend. On searching for "flaws" of the language and the platform, I (accidentally) ran into: https://www.youtube.com/watch?v=42k70Y-yTYY . (year 2017) Video description: ...What many developers don’t understand is that Erlang is built on an architecture and within ecosystem that contains many subtle secu…

This is an issue that’s overblown, simply have a firewall that exposes used ports and you’re fine.

There’s also an option within the release tool distillery to limit it to the local network.

Not to mention even if an attacker found your unsecured setup, they’d also need to know your “cookie”/key to do anything. It’s no different than leaving SSH login with password enabled on any server.

Post reply on HN