Live data from Hacker News

How We Went from 30 Servers to 2: Go

blog.iron.io

71–80 of 511 posts

Re: How We Went from 30 Servers to 2: Go

#71

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go. We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far. Lastly, for anyon…

Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.

Re: How We Went from 30 Servers to 2: Go

#72

Earlier quoted context omitted.

For my (toy) apps, I've been compiling locally and pushing to the server. It's trivial to compile for a target platform and architecture, and you get a single compiled binary. Certainly my response isn't getting all the way to what you're looking for, but it's not difficult to write shell scripts that manage the deploy process from here.

Thanks, I use rsync wrapped in a script usually (because I'm dealing with more files than just a binary), haven't tried cross compiling yet but was going to give that a go. I suppose I was more concerned about what happens when things are on the server and how multiple processes can be managed. As soon as you have more than one and want to swap out server workers seamlessly and load balance etc it gets a little more…

Yes, I think you're right on the mark re: swapping out workers, etc. Those nuances I haven't had to deal with due to these being toy apps.

Re: How We Went from 30 Servers to 2: Go

#73
post #6

Earlier quoted context omitted.

Mostly because I can't stand Javascript. It makes me cringe just thinking about it. Go is much nicer to work with.

Hey OP! I appreciate your sharing. Since you came from ruby and we're on the topic of the language itself, I'd appreciate your impression of how well Go supports collections. Is there or could one write something like http://underscorejs.org/ ? Can you do this kind of thing? [1, 2, 3, 4, 5].reject {|i| i I did the go tutorial the other day and I became a little worried that one would not be able to do this kind of th…

Go has first-class functions, so you could write a library do the same thing with, yes, a few more declarations around it. In your example, .map { |i| i + 9 } is .map( func(i int) int { return i + 9 }), or .map(myFuncDefinedElsewhere) in most real-world scenarios.

As a commenter upthread pointed out, Go and serverside Javascript are aimed at very different use cases and audiences.

Re: How We Went from 30 Servers to 2: Go

#74
post #30

> "We also weren't sure if we would be able hire top talent if we chose Go, but we soon found out that we could get top talent because we chose Go." I feel[1] that a smart/talented C/C++/anything developer can go from someone who has never seen or heard of golang to a proficient and productive Go developer in a matter of a few weeks, maybe even _days_, if not less. That's how long it takes to go through the following…

It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.

This is why Joel Spolsky correctly observed that the replacement of C/C++ and functional languages with Java in university CS curricula is a tragedy. If you don't understand pointers or recursion, you are not a polyglot and you cannot pick up just any language in a matter of weeks.

The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer talent.

Re: How We Went from 30 Servers to 2: Go

#76
post #71

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go. We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far. Lastly, for anyon…

Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.

Sure, check out their github pages (here's a few):

* https://github.com/mozilla-services/heka-mozsvc-plugins

* https://github.com/mozilla-services/heka

My friend works for Mozilla - that's how I knew.

Re: How We Went from 30 Servers to 2: Go

#77
post #71

Go's definitely picking up momentum. I know that Mozilla is shifting much of its services infrastructure to Go. We're porting over our Arabic sentiment engine, currently in Python/Cython to Go. If you're dealing with simple data structures, going from Python to Go is almost a line-for-line port, but the performance benefits are, of course, massive. Our benchmarks show a 40x speed improvement so far. Lastly, for anyon…

Do you have a citation for Mozilla using Go? I would be a bit surprised given their(Mozilla) development of Rust.

Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.

Re: How We Went from 30 Servers to 2: Go

#78
post #41

It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made. frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale. It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "th…

Nice rant, and I mostly agree with it, but there are a few slow things about the ways that some languages are usually implemented that don't really have much to do with over-engineering. Let's take this line of Python code, for example, and look at how the CPython interpreter runs it:

    return x + 42
This turns into the following bytecode instructions:

    LOAD_FAST       0 (x)
    LOAD_CONST      1 (42)
    BINARY_ADD
    RETURN_VALUE
First we get the x variable from the local namespace. This is an array of PyObject pointers; the LOAD_FAST oepration simply does an array lookup and puts the result on the stack, incrementing the reference count. Pretty fast. Next is LOAD_CONST, which is even faster; it takes an already-allocated PyObject and puts a pointer to it on the stack, incrementing the reference count. BINARY_ADD removes two numbers from the stack, dereferences the pointers to get the integer values, allocates a new PyObject with the resulting integer inside, bumps up its reference count, decrements the reference count of the two operands, and pushes the result on the stack. Finally, RETURN_VALUE jumps back to the caller of the function.

In Go, the corresponding code would be compiled to two, maybe three machine code instructions. There are good reasons why Python does it this way, but it does suffer some inherent slowdown.

Re: How We Went from 30 Servers to 2: Go

#79
post #6

Earlier quoted context omitted.

Mostly because I can't stand Javascript. It makes me cringe just thinking about it. Go is much nicer to work with.

Hey OP! I appreciate your sharing. Since you came from ruby and we're on the topic of the language itself, I'd appreciate your impression of how well Go supports collections. Is there or could one write something like http://underscorejs.org/ ? Can you do this kind of thing? [1, 2, 3, 4, 5].reject {|i| i I did the go tutorial the other day and I became a little worried that one would not be able to do this kind of th…

If you're looking for more elegant methods of expressing this in a higher performance runtime, here are two examples.

Erlang: [I+9 || I = 3] Haskell: [i+9 | i = 3]

Like go, both Erlang and Haskell both have strong concurrency stories. For an I/O bound server in a production environment, Erlang is a very safe choice.

Compared to these list comprehensions, that ruby example looks unwieldy.

Post reply on HN