Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Went from 30 Servers to 2: Go

blog.iron.io

Post reply on HN
Print view View original

51–60 of 511 posts

Re: How We Went from 30 Servers to 2: Go

#51

Post by laumars »

carbocation wrote: post #44
laumars wrote: post #40

Earlier quoted context omitted.

Thanks for that. The issue I have is that I would prefer to keep Apache around if I can (I have a few sites - some of which I host for friends). While my web server does have a few IPs attached, I'd rather not have to buy more IPs just to separate Apache from Go. And to be perfectly honest, I do quite like Apache. (each to their own I know, but I've had little reason to complain about it).

Right, you're describing my situation exactly. I used to run Apache solo. Now I run Nginx, which forwards requests to Apache or to my various go apps based on the domain name. Apache and the go apps just listen on different ports internally. The server only has 1 IP address.

You are an absolute star. If I could up vote you a hundred times I would.

Thank you

Re: How We Went from 30 Servers to 2: Go

#53

Post by mrwnmonm »

thibaut_barrere wrote: post #49
mrwnmonm wrote: post #24

man, you just made me worry very much now - we are making taxi dispatching api using ROR - and we will lunch it soon

Why worry? If you can handle the initial load, you can still optimize later on, rewriting some functions or parts of the system with something faster.

great

Re: How We Went from 30 Servers to 2: Go

#54

Post by bad_user »

jewel wrote: post #7

For large projects, it'd probably make more sense to rewrite the critical code sections in C. rubyinline makes this really easy, and ruby C extensions aren't that bad either. One time on a contracting gig I had some code that was interacting with cairo drawing and imagemagick. They didn't have a compatible raw image format at the time (one was RGBA and the other was ABGR, IIRC.) It's trivial to convert between the tw…

For web servers there are other things more important than the raw processing speed of a single routine. What you want is to get the CPU usage to 100%, CPU resources used preferably in actual processing and less on things like garbage collecting or blocking for I/O. For web servers in particular it is tricky since processing requests also involves a fair bit of I/O interwoven with CPU processing (in the case of apps that aren't just CRUD over a DB).

Re: How We Went from 30 Servers to 2: Go

#55

Post by jmilkbal »

thirsteh wrote: post #26
jmilkbal wrote: post #21

Earlier quoted context omitted.

It could also be said that developers who use established languages are excited to get things done rather than reinvent wheels continuously.

No, it can't. People don't use only Java because they "just want to get things done." They use only Java because that's all they learned, and all they need for their day job. If not, then they will invariably step over languages that are very interesting but obscure.

I'll give an anecdote. I've spent the last 7 years using Ada, a language that is /both/ interesting and obscure, almost exclusively. It's really a fantastic language that is frequently neglected much like poor Go, but with more merit. Anyway, in preparing the launch of my company's first iteration of its podcast network (http://76streetnetwork.com), I absolutely wanted to use Ada, but the allure of the utterly fantastic tool support and breadth of libraries available for Java had me up and running in a day or two despite having never written anymore than basic hello world-like crap with it in the past. There's really nothing crazy about Java, you're right, but I live for coding and I have a deep interest in quality software development, but I guess I'm just looking for a paycheck for this side project that will never make me any money! Ironically, Ada is what pays my bills.

Re: How We Went from 30 Servers to 2: Go

#56

Post by genwin »

laumars wrote: post #25

Has anyone had an experience of running websites from Go; or more specifically, how you handle none-HTML content? I've been considering porting my CMS from mod_perl to Go, but I'm not sure how you work with the other files (CSS et al). I did read somewhere that you run Go from Apache but it's not recommended.

My website, niflet.com, is written in Go. It uses Go's web server. While in learning mode for Go I saw that others were using Nginx instead. Now that I've gone through the development process I can see why. Go's web server is basic. Things like compressing files, setting header values and other niceties you have to write yourself or look for code written by contributors [1]. It's not as bad as it sounds though. Go has a function for serving static files like CSS files, which I wrapped in a function to serve a zipped version if the client supports that.

If I had it to do over again I would still use Go's web server, as it seems to work great. My site can conservatively handle 1 million page views hourly, using a cheap box.

[1] https://groups.google.com/forum/?fromgroups=#!topic/golang-n...

Re: How We Went from 30 Servers to 2: Go

#57

Post by memset »

TillE wrote: post #30
shurcooL wrote: post #20

> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go." I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less. That's how long it takes to go through the following…

It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.

I agree with the premise that, as a professional software engineer, it is my responsibility to be a polyglot.

As for myself, when I started writing Python, I mainly wrote C or Java code in Python - in a similar way, perhaps, how early C programs were littered with __asm__() constructs.

It took a long time to learn how to write things in (though I hate the term) a "pythonic" way. That is, to learn the common language idioms that are not taught in any tutorial, or are part of pep8, which illustrate common patterns in Python code, the ins-and-outs of PYTHONPATH, etc etc.

So while I agree that, in a weekend, a reasonably proficient programmer can pick up reasonable proficiency in a given language, being a "Python Programmer" to me means that one has developed an intuition for the common patterns, libraries, pitfalls, platforms, and clever specific features of a given language and its ecosystem.

Re: How We Went from 30 Servers to 2: Go

#58

Post by georgeorwell »

TillE wrote: post #30
shurcooL wrote: post #20

> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go." I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less. That's how long it takes to go through the following…

It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.

What, you don't know about The Programmer Hierarchy?

https://news.ycombinator.com/item?id=1622553

http://lukewelling.com/wp-content/uploads/2006/08/programmer...

I find a lot of people who use high-level languages are terrified of tediously direct contact with the machine, and a lot of people who use low-level languages are terrified of the performance costs of abstraction. I'm terrified of both.

I think that in general, C++ is a lot harder to get your head around than C, being for all intents and purposes a superset, and an easily five times bigger one at that.

Re: How We Went from 30 Servers to 2: Go

#60

Post by trotsky »

siavosh wrote: post #13

I've read this domino-effect on server clusters on several HN postmortems and I've seen various flavors of this on our own web servers. I'd like to think there's a simple configuration in most servers that prevent 100% cpu utilization from taking place and preventing the server from telling its cluster that it's still alive. Anyone have any experience with this?

in linux cgroups (control groups) allow you to allocate resources to groups of processes, which makes it easy to ensure things like ssh or npre or your load balancer ping (as long as you run it out of band) still have cpu/mem even when your main server cgroup gets pegged
Post reply on HN