Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How And Why We Switched from Erlang to Python (2011)

engineering.mixpanel.com

Post reply on HN
Print view View original

121–130 of 150 posts

Re: How And Why We Switched from Erlang to Python (2011)

#121

Post by cellularmitosis »

andy_ppp wrote: post #109
cellularmitosis wrote: post #105

Earlier quoted context omitted.

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

The problem here is not understanding your database not about making a system in an unsupported and probably undeployable thing to prove how clever you are...

> "to prove how clever you are"

I see this a lot and I just don't get it. Is a mechanic who uses air tools trying to "prove how clever he is"?

Knowing more than one tool allows you to choose the right tool for the job. Assuming arrogance and a motive of "proving cleverness" sounds like a great way to pick a fight.

Re: How And Why We Switched from Erlang to Python (2011)

#122

Post by shRaj9fEc8Vith »

kvakvs wrote: post #30
robertAngst wrote: post #26

Earlier quoted context omitted.

>Pick a language that has limited programmers, bring programmers from around the country, pay them well beyond 6 figures so they don't leave and have a crisis every time a programmer leaves. OR >Go with a mainstream language that they teach every CS kid and has every googable question imaginable in SO, hire 1 experienced programmer to manage a bunch of post-college-kids. There are lots of good reasons to pick a langu…

You do not need a large team of Erlang developers though, a much smaller team can hold the project afloat, even hire consultants to fix it and leave it for a few years. Also Erlang dev salaries aren't that stellar, just average what you'd pay a Python or a Java person.

it's almost impossible to find Erland dev in my country :D

of course we can have someone learn and then teach to the rest of the team but why bother doing that when you can select another language where it's a lot easier to hire

Re: How And Why We Switched from Erlang to Python (2011)

#123

Post by bdibs »

dhab wrote: post #120
bdibs wrote: post #110

Earlier quoted context omitted.

This is an issue that’s overblown, simply have a firewall that exposes used ports and you’re fine. There’s also an option within the release tool distillery to limit it to the local network. Not to mention even if an attacker found your unsecured setup, they’d also need to know your “cookie”/key to do anything. It’s no different than leaving SSH login with password enabled on any server.

The speaker claims that the cookie/key can be known by causing VM crash, which is done by exploiting the monotonic nature of cookie values which are stored as atoms which has a fixed cap on how many they can be, which when exceeded causes VM to crash. He estimates the time to do that be 10 mins (I forgot the exact memory limit)

That’s definitely an issue, but again can be easily mitigated with a simple firewall that doesn’t allow every port to be open to the world.

Re: How And Why We Switched from Erlang to Python (2011)

#124

Post by cellularmitosis »

theamk wrote: post #108
cellularmitosis wrote: post #105

Earlier quoted context omitted.

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

If there are no other people who can maintain that stack, maybe (firing is too extreme, I’d just ban the Clojure code in the production) Writing initial version of the code is only a small part of it. Maintaining and enhancing it is much harder. What is the company going to do if the new endpoint went down and the author is on vacation?

> just ban the Clojure code in the production

A dev ran up against the limitations of one platform and solved the problem by choosing a platform which didn't have those limitations. And management responded by banning the new platform.

How long do you think that dev is going to stick around in an environment like that?

Ultimately this comes down to what sort of environment you want to foster. One approach leads to an environment where devs have lots of freedom, and that creates certain problems. The other approach eventually leads to an environment where even the choice of editor / IDE is dictated, and that creates a different set of problems.

I'd favor attracting a bunch of smart folks and give them as much freedom as possible. I'd much rather deal with the kind of problems which that sort of environment creates.

Re: How And Why We Switched from Erlang to Python (2011)

#125

Post by verinus »

cellularmitosis wrote: post #121
andy_ppp wrote: post #109

Earlier quoted context omitted.

The problem here is not understanding your database not about making a system in an unsupported and probably undeployable thing to prove how clever you are...

> "to prove how clever you are" I see this a lot and I just don't get it. Is a mechanic who uses air tools trying to "prove how clever he is"? Knowing more than one tool allows you to choose the right tool for the job. Assuming arrogance and a motive of "proving cleverness" sounds like a great way to pick a fight.

this is not a valid comparison.

a mechanics' tools would be IDE, build tools etc- I'm not interested how every dev uses personal level tools.

But just writing a program with a new dev stack without thinking of the possible consequences is just plain unprofessional.

Re: How And Why We Switched from Erlang to Python (2011)

#126

Post by _pmf_ »

cellularmitosis wrote: post #105
bradleyjg wrote: post #54

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

> lol yeah that guy should totally be fired.

Yeah, introducing a new ecosystem to mask bad DB design or bad queries because she is lacking basic SQL and relational foundations is how I envision the typical 10x programmer.

Re: How And Why We Switched from Erlang to Python (2011)

#127

Post by YeGoblynQueenne »

>> I’ve learned a lot about how to scale a real service in the couple of weeks I’ve been here.

I take this to mean that the new server implementation in Python was written in at most two weeks by a developer who was an intern at the time. I don't think I'd be easily convinced that best results could be obtained this way.

The original Erlang implementation could be flawed and buggy, but wouldn't a much better result be achieved by fixing its problems, rather than rewriting it from scratch to another language, that is not known for its efficiency or for being very good for networking, unlike Erlang?

In any case- is there ever a case where an intern can write the best possible system in two weeks? And why would you hand off a critical part of your system to the most junior member of the team? I mean, unless your team are all elite hackers from the higher echelons of computer science masters and the "most junior member" is one of the best programmers in the world, or so?

Re: How And Why We Switched from Erlang to Python (2011)

#128

Post by collyw »

cellularmitosis wrote: post #105
bradleyjg wrote: post #54

Earlier quoted context omitted.

In general language diversity at a company is a big negative. Obviously sometimes there are even bigger negatives that mandate overriding that rule--you aren't going to use Java on the front end or javascript on the backend, that would be madness--but you never want two languages in the same niche. That's a cost with no benefit. If you have some developer that wants to write something in a new (for your company) lang…

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

Rather than making thousands of requests in parallel, wouldn't reducing the number of queries be the more obvious choice.

Re: How And Why We Switched from Erlang to Python (2011)

#129

Post by collyw »

_pmf_ wrote: post #126
cellularmitosis wrote: post #105

Earlier quoted context omitted.

Some dev: "Hey, I noticed one of our more complex API endpoints had some pretty bad response latency, so I dug into it a bit. It turns out our PHP/Symfony/Doctrine stack was generating over a hundred SQL queries, all of which were being performed sequentially because we don't have a good per-request concurrency story in this tech stack. So I wrote an alternate implementation in Clojure which has a great concurrency s…

> lol yeah that guy should totally be fired. Yeah, introducing a new ecosystem to mask bad DB design or bad queries because she is lacking basic SQL and relational foundations is how I envision the typical 10x programmer.

I hope you are being sarcastic.

Re: How And Why We Switched from Erlang to Python (2011)

#130

Post by collyw »

bradleyjg wrote: post #104
tomjakubowski wrote: post #86

Earlier quoted context omitted.

You could fire them or you could just help them understand something that they don't. Sheesh.

Fair. That part was over the top. But I do think employees pushing their own intellectual passions or resume building pursuits over the interests of their employer are pernicious. If it’s out of ignorance, then sure by all means educate.

It's a two way street. Employers always want the most up to date trendy frameworks on their employees CV's. I imagine someone that has been working on a legacy VB6 app for the last few years will struggle to get his CV looked at.
Post reply on HN