Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Went from 30 Servers to 2: Go

blog.iron.io

Post reply on HN
Print view View original

361–370 of 511 posts

Re: How We Went from 30 Servers to 2: Go

#361

Post by coldtea »

jbooth wrote: post #207
azakai wrote: post #77

Earlier quoted context omitted.

Rust and Go are not similar or competing. They are both new, but aside from that the differences are huge. So if a project uses one, that doesn't mean the other was a possibility for that project too.

They're not similar at all, but both were designed to replace C++. That sort of means that they're competing for the use-case of "things I would write in C++ if it wasn't a god-awful nightmare to do so". From there, though, you're right that they take very, very different approaches.

>They're not similar at all, but both were designed to replace C++.

Well, Rust WAS designed to replace C++, both in the intention behind its design and the way it was done.

Go designers had this vague intention of "replacing C++", but the way they have designed the language they only really replace Java or some scripting language. Which they admit (the get mostly Python etc converts than C/C++ converts).

Re: How We Went from 30 Servers to 2: Go

#362

Post by hvidgaard »

quaunaut wrote: post #86
zobzu wrote: post #41

It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made. frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale. It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "th…

Maybe I'm just a new and terrible programmer, but I feel like if you wait to fully understand whatever tech is out there in terms of what to use, you'll never get started.

In reality you have to follow the buzz in the tech to understand new technology. If something sounds like you could use it, take a few days out to toy with it. If you still want to use it, try it out for a small project and the rest will come automatically if it is the right choice.

Re: How We Went from 30 Servers to 2: Go

#363

Post by qznc »

mercurial wrote: post #340
qznc wrote: post #291

Earlier quoted context omitted.

I once heard a quote like this "programming languages are frozen knowledge about software engineering". (Anybody got a clue who said something like this?) New languages usually improve upon older languages by making certain errors impossible by design. For example, Go allows pointers, but not pointer arithmetic. With C we learned pointer arithmetic is often harmful (though sometimes necessary), so Java removed pointe…

Funny you should take Go as an example, since it's often bashed for not having taken the last decade of language design into account.

Go does take the last decade of language design by Pike, Thompson, and Griesemer into account. Mostly Pike as I am not sure if the other two guys even did design some language in this time.

Personally, I consider gofmt the biggest achievement of Go, if it manages to make that mainstream. While there are equivalent tools for C they are not widely used.

Re: How We Went from 30 Servers to 2: Go

#364

Post by Peaker »

masklinn wrote: post #320
Peaker wrote: post #146

Earlier quoted context omitted.

Haskell's runtime makes for cheaper threads than Erlang. Possibly cheaper than Go's, too.

> Haskell's runtime makes for cheaper threads than Erlang. Possibly cheaper than Go's, too. What's your measure for expenses? Because if it's size in memory goroutines seem more expensive than erlang processes: http://en.munknex.net/2011/12/golang-goroutines-performance.... indicates goroutines were measured at ~4k; the default start size for an erlang process is ~310 words, or ~2k on a 64b machine.

1kb in ghc, iirc.

Re: How We Went from 30 Servers to 2: Go

#365

Post by masklinn »

qznc wrote: post #363
mercurial wrote: post #340

Earlier quoted context omitted.

Funny you should take Go as an example, since it's often bashed for not having taken the last decade of language design into account.

Go does take the last decade of language design by Pike, Thompson, and Griesemer into account. Mostly Pike as I am not sure if the other two guys even did design some language in this time. Personally, I consider gofmt the biggest achievement of Go, if it manages to make that mainstream. While there are equivalent tools for C they are not widely used.

> Go does take the last decade of language design by Pike, Thompson, and Griesemer into account.

Decade which ended 20-odd years ago relative to the rest of the world, kind-of the point.

Re: How We Went from 30 Servers to 2: Go

#366

Post by lifeisstillgood »

philh wrote: post #333
lifeisstillgood wrote: post #285

Earlier quoted context omitted.

Could you give a specific example as I am still thinking "random number generator" and want to balance it with your "standard idiom or library that everyone steeped in $lang knows " Cheers

I dunno how well-known the things I didn't know about are, but an example of me doing it: https://news.ycombinator.com/item?id=5113202

I think there is a fine line between using a library and writin your own to understand better the domain

It's something to do with how critical the library functions are to you / your system. I would never write my own compression software, but I can see why people would just to learn about the trade offs.

Re: How We Went from 30 Servers to 2: Go

#367

Post by pjmlp »

codygman wrote: post #341
pjmlp wrote: post #311

Earlier quoted context omitted.

Same thing happen to me, I even did some small contributions around 2010. Then I discovered D and Rust are better languages for my purposes.

Just a guess, I'm guessing python isn't your favorite language either? They seem to be rather unopinionated languages that let you have access to lots of tools. I hear they manage to keep it pretty simple despite having a "kitchen sink", so definitely interested in both.

Python is my favorite scripting language, not for application development.

Re: How We Went from 30 Servers to 2: Go

#368

Post by pjmlp »

codygman wrote: post #338
pjmlp wrote: post #305

Earlier quoted context omitted.

Like any other language with static typing and available compiler implementations.

Is that relevant though? The parent is asking about a comparison between python+c and Go for the OP's software, so doesn't it make sense to answer to that? Of course, you are correct. Rewriting in any other language with static typing or an available compiler implementation would also provide the benefits of static typing and detection of runtime errors.

Because those benefits are not some special feature of Go.

On the other hand I think youth generation of programmers lack exposition to programming languages, like we used to have in the old days, hence they make quite limited comparisons.

Re: How We Went from 30 Servers to 2: Go

#369

Post by microtonal »

waterside81 wrote: post #141
cpprototypes wrote: post #111

Earlier quoted context omitted.

Just curious, what were the tradeoffs for profiling the Python code and rewriting slow parts in C vs total rewrite in Go? For high level languages, the usual argument has been to rewrite just the slow parts in C or some other low level language.

And that's what we did during our first pass: identify the slow parts and port them to C. This gave us about 30% better performance. Without getting too much into the nitty gritty of our specific case, this wasn't enough. We needed some massive speed improvements, I'm talking in the order of 100X faster. The nature of our algorithms was such that they could be done in parallel (i.e map/reduce) - an ideal candidate fo…

One other aspect that really guided us towards Go was memory usage. Python was just sucking up so much memory whereas our Go implementation thus far is so much thinner.

It's funny that you mention that, because in computationally intensive work memory management is an issue with Go. E.g. I wrote a maximum entropy parameter estimator in Go, which was terribly slow until I circumvented the garbage collector by preallocating a huge block of memory and doing my own management. In C malloc() and free()-ing had practically no overhead. After putting the Go garbage collector out of the game, the Go version was approximately within 2x of the C version.

I am interested how Go gave you one or two orders of magnitude speedup, while rewriting hot spots in C didn't...

Re: How We Went from 30 Servers to 2: Go

#370

Post by pjmlp »

kyllo wrote: post #74
TillE wrote: post #30

Earlier quoted context omitted.

It's quite strange to me that people would identify as or look for a "[language] programmer". Sure, I happen to write more C++, Python, and C than anything else, but I've dabbled in just about everything and could reach comfortable proficiency in a matter of weeks. Most of programming and all of computer science is universal. Any serious programmer should be a polyglot by default.

This is why Joel Spolsky correctly observed that the replacement of C/C++ and functional languages with Java in university CS curricula is a tragedy. If you don't understand pointers or recursion, you are not a polyglot and you cannot pick up just any language in a matter of weeks. The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer…

> The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer talent.

From what I can experience in the multi-site enterprise projects I participate, the industry is being quite successful at that.

Post reply on HN