Earlier quoted context omitted.
> C++ and Python pretty much run everything, everywhere. I don't think so. Last I heard, there was still a lot of COBOL out there.
COBOL gets all the hate but if they have a Big iron mainframe then they'll most likely be running RPG.
How We Went from 30 Servers to 2: Go
441–450 of 511 posts
Re: How We Went from 30 Servers to 2: Go
#442Earlier quoted context omitted.
Rails is pretty crappy if you ask me. It's "omakase" which is Japanese for "acts according to what DHH wants despite what the community wants". And there's a lot of magic happening that isn't explained very well. There are better frameworks in Ruby. Ruby itself doesn't take all that long to be an expert at.
Though I could argue that all the languages pretty much take the same amount of time to become an expert. Just because some languages are supposedly higher-level than others doesn't mean that the complexity they allow you to tackle (and associated challenges as a programmer or "expert") is any less. Especially since being a good programmer is more about design, choice of interfaces, reactivity to change, etc. which a…
Re: How We Went from 30 Servers to 2: Go
#443Earlier quoted context omitted.
Why would you say something like this? The IRC Python channel is a kind of state machine which goes like: - How do I do this? - Why do you want to do this? - Because XXX - Then that's not really what you should be doing, it's dangerous/inefficient/etc., do this instead The post about Acme could have been a one-line answer: "If you want do do any of this, just use a better editor".
There are many times when I said "I need to do X, I know it's ugly, but I've considered all the alternatives and, for reasons too lengthy to go into right now, I just need to do X. How can I do it?" only to get people treating me like an idiot who doesn't know anything, and only giving an answer after I've responded to every single of the alternatives with my reasons. Sometimes, people aren't new.
Re: How We Went from 30 Servers to 2: Go
#444Earlier quoted context omitted.
Your inability to understand their clear description of a basic cascading failure mode under load speaks poorly of your actual knowledge and experience. Given that, I have to take everything else that you say with a large helping of salt.
Their description of the root problem was very superficial: > At some threshold above 50%, our Rails servers would spike up to 100% CPU usage and become unresponsive. Yes, but why? What exactly were those processes doing? Why the sudden change at a particular threshold? Their lack of detailed investigation into this makes their post useless to me -- I have no way of knowing (1) what specific aspect of Ruby's architec…
The bigger problem was dealing with big traffic spikes. When a big spike in traffic came in, it created a domino effect that would take take down our entire cluster
Given the article to that point, it is clear what is happening. They are maintaining a steady state of X% (where X is above 50), then a big traffic spike comes along. That traffic load is not distributed equally (my guess is because requests are not created equal) and there is a hot spot. The hot spot fails, then increases the load on everything else. After this repeats a few times, there is insufficient capacity.
In other words at some steady state threshold above 50%, you wind up without sufficient capacity to handle traffic spikes. Nothing in this failure scenario is specific to Rails. It is a well-known failure mode for a cluster under load with a push based architecture (which http requests are).
The fact that you did not understand that description speaks poorly of your problem solving skills. Now they may well have been doing something trivial that is fixable to cause load to be less than it was. But you haven't convinced me that you're the person to be trusted to figure that out.
Re: How We Went from 30 Servers to 2: Go
#445Earlier quoted context omitted.
It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.
If only the morons that hire could figure this out. I almost NEVER use Java, but obviously I have and CAN use Java. Yet interview after interview demands that I use Java day to day in my current job to be considered. And this is for data transformation/analysis jobs, in which my use of Python makes me dramatically faster and more productive than my Java using colleagues.
Re: How We Went from 30 Servers to 2: Go
#446Earlier quoted context omitted.
> but real motives are hidden Could you elaborate on this please? Are you referring to the OP's motives of publishing the article? Or of switching to Go? > Go and Rails are so different that there is almost no point in comparing them. Except when one solves the same problem better than the other.
- They did not say why they did not choose one of the other alternatives. - They did not say how they used the concurrent features of Go, only mentioned that they were there. - They did not say how long it took to rewrite. - They did not say what they changed in the API. I'm not saying the story isn't true, but for a true story it lacks a lot. You could summarize the article with the title and you won't be missing mu…
Instead, it's "OMG we're maxing out CPU, time to completely rewrite our app in a different language"
And how big is this app that it needs 30 servers? We probably serve an order of magnitude of traffic where I work than these guys do (just guessing) and don't need that many.
Re: How We Went from 30 Servers to 2: Go
#447Has anyone had an experience of running websites from Go; or more specifically, how you handle none-HTML content? I've been considering porting my CMS from mod_perl to Go, but I'm not sure how you work with the other files (CSS et al). I did read somewhere that you run Go from Apache but it's not recommended.
I've just started creating toy projects in Go. I put these Go projects (and Apache, which runs older PHP projects) behind nginx. For hosting CSS, you either generate it programmatically and send the right Header, or you can use http.FileServe from the standard library [1]. (Surely other approaches are possible, but those are the two I've played with so far.) [1] https://code.google.com/p/go-wiki/wiki/HttpStaticFiles
Re: How We Went from 30 Servers to 2: Go
#448> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go." I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less. That's how long it takes to go through the following…
It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.
The language itself is not usually a problem for me. I mean an array is an array in any language. It's environment. Example: c# (which I think in MS's best work, by the way); c# was easy to pick up. The .Net framework was another matter. That took time. Knowing where the resources are and hell - even getting comfortable with the documentation and the IDE took time.
So now, when I hire a programer, I'm not too interested in his language skills. I'm assume he can code well enough not to embarrass himself. It's his knowledge of the environment that we run. That's what I look for.
Re: How We Went from 30 Servers to 2: Go
#449Earlier quoted context omitted.
> Any serious programmer should be a polyglot by default. It depends if you're going to spend years training someone or if you need an expert right now. My experience is that it is impossible to maintain expert level skills in more than one or two language + library environments. You can remain familiar with other environments but you don't have the time to be an expert. While I sometimes switch between C-family lang…
However, you do not need to be an expert to be productive. You can write Django web apps perfectly well without knowing about Python's metaclass tricks, for example. An expert still needs time to become familiar with a large codebase and architecture. I wonder if a competent programmer could simply learn the language in addition during this period.
Re: How We Went from 30 Servers to 2: Go
#450I was originally very excited about Go when I first learned about it. But then I got tired and frustrated quickly after having to listen to the other Gophers telling me that I don't need this or that feature because there is a better way to do it in Go. Like. I don't need exceptions because Go function can return multiple values. I don't need a mocking framework like Mockito because Go has interfaces. I don't need an…
We also checked out both Go and Scala and picked Scala due to better IDE support (IntelliJ) and existing Java lib ecosystem. It also is typically faster than Go (but does use much more memory): http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t... Overall very happy with our choice. I will probably learn Go too, but I also don't really like the way it does error handling and the tooling support seems limite…