Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How We Went from 30 Servers to 2: Go

blog.iron.io

Post reply on HN
Print view View original

391–400 of 511 posts

Re: How We Went from 30 Servers to 2: Go

#391

Post by taeric »

iforgotagain wrote: post #374
taeric wrote: post #165

Earlier quoted context omitted.

Just no. Some abstractions are light years faster than not having them. To the point that this is a silly statement. Consider the elementary abstraction of an array. The slightly less elementary one of a map. More close to heart, the abstraction that source control gives you. Imagine not having git (or whatever tool) and thinking you will compare this weeks code with what was available last week. Now just do the same…

>> Just no. Some abstractions are light years faster than not having them. To the point that this is a silly statement. Consider the elementary abstraction of an array. The slightly less elementary one of a map. Abstractions can definitely be an improvident in some cases. They just bring extra complexity at it's cost. In the 'map' case you pointed out, you now need to take into account the performance characteristics…

But that is my point, if used correctly, the abstraction just makes it easier to do what you want to do quickly. Consider, math is basically abstracting away the tedious nature of some physical process. I mean, if you know it takes 2 eggs to make 4 waffles, and want to make 82 waffles, the abstraction of math makes this much quicker than just doing it to determine how many eggs you need.

Basically, I object to the idea that abstractions slow you down by their nature. I agree that poor or unnecessary ones do so.

Re: How We Went from 30 Servers to 2: Go

#392

Post by ralph »

tjgq wrote: post #389
pjscott wrote: post #147

Earlier quoted context omitted.

Nope! There's a branch in the interpreter that checks if both operands are Python's built-in int objects. If they are, and the result can fit in a C int without overflow, then the interpreter adds the numbers directly. This is by far the most common case.

How can the interpreter know, at bytecode compilation time, that x is bound to an int? Surely it generates two code paths, one for the "common case" and another for when x is a general object with an __add__ method?

The test is in the interpreter, not the compiler to byte-code. Every time the + is evaluated, the interpreter checks for the common case before resorting to finding the first object's method.

Re: How We Went from 30 Servers to 2: Go

#393

Post by Shamharoth »

qznc wrote: post #287
eru wrote: post #222

Earlier quoted context omitted.

> I like how Lisp and Assembler at the top of the hierarchy capture the two extremes. Not really. I'd place Agda or Coq above Lisp.

That is somewhat orthogonal. Lisp is better at abstraction than Agda or Coq or Isabelle or any of those ML/Haskell theorem proofers. To maintain the theme: C if you are terrified about performance Lisp if you are terrified about boilerplate Agda if you are terrified about correctness If you are terrified about all of these, then welcome to the world of engineering.

If you are terrified about all these, there is ADA.

Re: How We Went from 30 Servers to 2: Go

#394

Post by Xion »

redbad wrote: post #323
trungonnews wrote: post #232

Earlier quoted context omitted.

A seasoned Java developer is expected to have worked with one of the mainstream ORM like Hibernate or Ebean. But a Go developer gets away with not having to know an ORM because there is no mainstream ORM in Go. :)

ORMs are broadly considered antipatterns in Go.

Source? I have never heard that claim. And contrary to that, Go has facilities for implementing ORMs, like field annotations in structs for specifying column bindings.

Re: How We Went from 30 Servers to 2: Go

#395

Post by smosher »

rektide wrote: post #112
zobzu wrote: post #41

It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made. frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale. It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "th…

I'm very sympathetic to your tech-on-tech-on-tech-on-tech hogwashery promulgated by open sourcers & vendors prosteltyzing their wonderful solutions and the other acculturation factors that permits developers approaches other than trying & seeing & exploring- That said, you are wrong. I'm not a Go user and I don't care for it, but Go is fundamentally different and better than most runtimes. Go has it's own green threa…

Go is... fundamentally different than most runtimes? Probably. Better? Maybe. Better than the comparable ones? I think it's losing that war. You don't even need a special runtime, it can help, but I think the amount it helps is overstated in Go's case.

Concurrency isn't that hard in practice. If you're willing to forgo automatic multiplexing Perl will give you Go's concurrency model in a library written ten years ago, and it works as naturally as it does in Go. Python seems to have similarly good support but it probably doesn't look much like Go. I'm not sure what Lua's doing these days, but in the past I've found concurrency dead simple. If you happen to be writing C, time-slicing the work isn't even that hard. If you know enough about Go's concurrency to tell whether a single goroutine will cause its scheduler to deadlock all by itself, you can easily write concurrent programs in all these languages.

I know this will be a controversial claim, but all of these solutions are easier to reason about than Go. Usually not by much, and it has primarily to do with the nature of its multi-threading support, but that's just the concurrency model being weakened by the support for convenient parallelism.

There is a case where Go is best: when you like writing Go more than the others. You can guarantee a little less about concurrency than most, but at least you can usually guarantee more about types than most. If you like writing Go you will either luckily avoid the problems or develop habits that prevent you from producing concurrency bugs, possibly even without realizing it.

Re: How We Went from 30 Servers to 2: Go

#396

Post by ajanuary »

"So then the decision came down to which language to use."

I find this tendency to constantly jackknife between talking about languages and talking about frameworks a little dizzying.

Shouldn't the decision have been what web stack to use? I'm sure you could get half way there with a bespoke Ruby web stack, maybe built on top of JRuby or with a sprinkling of C extensions. You could get the same poor scalability with some badly designed framework on top of Go [1].

There's a lack of meat in the post (hence why people are calling it out as PR). How do these features of Go make it easier to write applications or frameworks that cater to this particular scalability need? Could extra effort be put in up front to get something similar out of Ruby or any of the other languages, or will they always be sub-par? What is it about Go that makes it light up the rest of the stack in a way other languages don't?

Or have you just traded one trendy technology for another because it promises to be the magic bullet for your current itch?

[1] This isn't saying Rails is badly designed.

Re: How We Went from 30 Servers to 2: Go

#397

Post by tjgq »

ralph wrote: post #392
tjgq wrote: post #389

Earlier quoted context omitted.

How can the interpreter know, at bytecode compilation time, that x is bound to an int? Surely it generates two code paths, one for the "common case" and another for when x is a general object with an __add__ method?

The test is in the interpreter, not the compiler to byte-code. Every time the + is evaluated, the interpreter checks for the common case before resorting to finding the first object's method.

Okay, I think I get it now. The check you refer to is done inside the BINARY_ADD implementation. That's why the bytecode is the same for both cases. Am I right?

Re: How We Went from 30 Servers to 2: Go

#398

Post by vidarh »

zobzu wrote: post #41

It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made. frameworks on top of frameworks, all being over engineered, with poor understanding of what the system actually does, result in super slow apps on top, that you generally go to aws to scale. It's not the first time that I see people reducing a dozen servers that were "always maxed out" by a couple of servers "th…

> It's not really "go" that makes the difference. it's how the runtimes and frameworks are used and/or made.

Exactly.

For the type of product they sell, if the processing time isn't pretty much all going to the actual running of the customer supplied job code and network latency, they're doing something very wrong. The language choice shouldn't be much more than noise in a setup like that. The framework choice, on the other hand... Rails is hardly well optimised for a high traffic API that can't readily be cached.

I've done queuing and job processing in Ruby. Spending 90% of the time in kernel space handling network traffic or waiting is not hard to achieve for a pure queuing server. If you add delegating jobs to external code to do the actual work, you should spend less time in userspace in the code actually processing the messages.

If language choice nets you more than 10%-20% in a setup like that, something else has changed beyond language.

And that's fine. But a post focusing on the language change then makes it seem like they didn't understand what caused the performance problem in the first place.

Re: How We Went from 30 Servers to 2: Go

#399

Post by kyllo »

pjmlp wrote: post #370
kyllo wrote: post #74

Earlier quoted context omitted.

This is why Joel Spolsky correctly observed that the replacement of C/C++ and functional languages with Java in university CS curricula is a tragedy. If you don't understand pointers or recursion, you are not a polyglot and you cannot pick up just any language in a matter of weeks. The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer…

> The "[language] programmer" (where [language] usually = Java or .NET) trend is a misguided attempt by industry to commoditize programmer talent. From what I can experience in the multi-site enterprise projects I participate, the industry is being quite successful at that.

Successful at commoditizing programmer talent? Perhaps.

Successful at producing high-quality software, relative to the resources they are expending on it? Not so much.

Re: How We Went from 30 Servers to 2: Go

#400

Post by pjmlp »

qznc wrote: post #363
mercurial wrote: post #340

Earlier quoted context omitted.

Funny you should take Go as an example, since it's often bashed for not having taken the last decade of language design into account.

Go does take the last decade of language design by Pike, Thompson, and Griesemer into account. Mostly Pike as I am not sure if the other two guys even did design some language in this time. Personally, I consider gofmt the biggest achievement of Go, if it manages to make that mainstream. While there are equivalent tools for C they are not widely used.

You mean by designing a language that is basically Alef from Plan9(1992) with a few changes?

Yeah, really actual.

Post reply on HN