A little Golang way • Hacker News Board

Re: A little Golang way

#171

Post by nindalf » Thu, Aug 06, 2015, 3:18 AM UTC

eropple wrote: post #73

nickbauman wrote: post #60

Earlier quoted context omitted.

Top notch tooling? The debugger is neigh unusable. `go get` is a community joke. The compiler is fast mostly because it doesn't check for things that better compilers do. Go race is a symptom that not all is well in the CSP house. Other static analysis tools are third party and limited if they exist at all. Go really does remind me of JDK 1.4.

I'm sad that your post is downvoted, because (while I might have been a little more pleasant about it) I think you're generally on-point. The debugger is pretty poor, the compiler is pretty poor, there isn't much static analysis (and while the language's youth is an excuse, the profusion of tools for more semantically rich languages like Scala make me skeptical of the excuse). I didn't mention Java 1.4 just for a lar…

Perhaps his post was downvoted because he made a strong assertion - "the compiler is fast because it doesn't check for things that other compilers do" without elaborating or providing any evidence. This is the first time that I've seen someone complain about deficiencies in the compiler too, so that adds a little skepticism.

Could you elaborate on the static analysis that you find lacking?

Re: A little Golang way

#172

Post by pcwalton » Thu, Aug 06, 2015, 3:20 AM UTC

Scramblejams wrote: post #167

pcwalton wrote: post #156

Earlier quoted context omitted.

Why do you need the threads to be "green"? That's what I'm asking. What's wrong with kernel threads, specifically?

In my experience, kernel threads have a cost that is sufficiently far from that of, for example, Erlang processes (400+ bytes each) that it changes the way you program with them. In Erlang you don't think twice about spawning a million threads if that models your problem nicely. The same isn't true when you're dealing with kernel threads. (Right?) So I'm interested in green threads because when I have to start thinki…

So since you're talking about scalability into the millions of threads, I think what you actually want is stackless coroutines rather than M:N threading with separate user-level stacks. If you have 1M threads, even one page of stack for each will result in 4G of memory use. That's assuming no fragmentation or delayed reclamation from GC. Stacks, even when relocating, are too heavyweight for that kind of extreme concurrent load. With a stackless coroutine model, it's easier to reason about how much memory you're using per request; with a stack model, it's extremely dynamic, and compilers will readily sacrifice stack space for optimization behind your back (consider e.g. LICM).

Stackless coroutines are great--you can get to nginx levels of performance with them--but they aren't M:N threading as seen in Golang. Once you have a stack, as Erlang and Go do, you've already paid a large portion of the cost of 1:1 threading.

Re: A little Golang way

#173

Post by mavelikara » Thu, Aug 06, 2015, 5:06 AM UTC

threeseed wrote: post #163

nine_k wrote: post #66

Earlier quoted context omitted.

This is true regarding heap space (you can allocate couple gigs in one line), but perm gen space and JIT space can also be significant in JVM case.

Permgen space doesn't exist in the JVM any longer. And not sure what JIT space is.

I think the OP meant "Code Cache" when they said "JIT space".

Re: A little Golang way

#174

Post by Scramblejams » Thu, Aug 06, 2015, 5:09 AM UTC

pcwalton wrote: post #172

Scramblejams wrote: post #167

Earlier quoted context omitted.

In my experience, kernel threads have a cost that is sufficiently far from that of, for example, Erlang processes (400+ bytes each) that it changes the way you program with them. In Erlang you don't think twice about spawning a million threads if that models your problem nicely. The same isn't true when you're dealing with kernel threads. (Right?) So I'm interested in green threads because when I have to start thinki…

So since you're talking about scalability into the millions of threads, I think what you actually want is stackless coroutines rather than M:N threading with separate user-level stacks. If you have 1M threads, even one page of stack for each will result in 4G of memory use. That's assuming no fragmentation or delayed reclamation from GC. Stacks, even when relocating, are too heavyweight for that kind of extreme concu…

Thanks for the tip, stackless coroutines are new to me. Any of that on the Rust roadmap?

Re: A little Golang way

#175

Post by vonsnowman » Thu, Aug 06, 2015, 7:16 AM UTC

mwcampbell wrote: post #145

vonsnowman wrote: post #140

Earlier quoted context omitted.

> This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons. > [...] while an app written in C++, Rust, or Go will take 2MB. Agreed. As mentioned in the blog post, I considered Rust but decided against it because of I found it much less mature than Go. I did not consider C++ because, as mentioned in the blog post, part of the poi…

I assume you didn't use the normal Dockerfile-based build system to build your super-small Docker images for the Go-based services. So how did you do it?

We're not doing anything too fancy. Basically, we spawn a container to build a statically linked binary and do a regular Dockerfile-based build inside that container. The result is an image which contains only a single binary (any maybe some static assets like config files or images).

We're planning to open source our build script shortly.

Re: A little Golang way

#176

Post by Scramblejams » Thu, Aug 06, 2015, 8:50 AM UTC

Scramblejams wrote: post #174

pcwalton wrote: post #172

Earlier quoted context omitted.

So since you're talking about scalability into the millions of threads, I think what you actually want is stackless coroutines rather than M:N threading with separate user-level stacks. If you have 1M threads, even one page of stack for each will result in 4G of memory use. That's assuming no fragmentation or delayed reclamation from GC. Stacks, even when relocating, are too heavyweight for that kind of extreme concu…

Thanks for the tip, stackless coroutines are new to me. Any of that on the Rust roadmap?

Am I correct that coroutines are intrinsically non-preemptive? If so, I'll need to keep looking.

Re: A little Golang way

#177

Post by stesch » Thu, Aug 06, 2015, 8:53 AM UTC

patio11 wrote: post #32

stesch wrote: post #18

How tied is Go into Google? What if Google drops Go?

You're probably being downvoted for off-topicness, FYI, since this comment is equally germane to any mention of Go. That said, I feel it is a comment in good faith and deserves an answer. Question #1: How invested should you model Google as being in Go's future? Answer: Extraordinarily invested. They have a whole lot of code written in it, most of which the world will never hear about, and which powers services that…

I'm not sure if this will help me. I guess I'm in a programmer midlife crisis. Every new thing is potentially a waste of time because you can't predict if it is still around in 2 years. And with Google you know that they invest millions of dollars in projects and then abandon them.

Would I be better off maintaining decade old COBOL projects? Sometimes I think so. Other times I'm glad I can chose the technology I want to solve the problems at work.

August 2015. 31 years since my first computer.

Re: A little Golang way

#178

Post by pron » Thu, Aug 06, 2015, 9:02 AM UTC

skj wrote: post #161

threeseed wrote: post #159

Earlier quoted context omitted.

Go's channels/threading model already exists in Java: https://github.com/puniverse/quasar

It wasn't immediately obvious from scanning that link. Does it provide channel select? Because that's the hard part. Threadsafe queues are easy, select on them is hard.

Of course it does.

Java: http://docs.paralleluniverse.co/quasar/javadoc/co/parallelun...

Clojure: http://docs.paralleluniverse.co/pulsar/api/co.paralleluniver...

We'll have a nice slect API for Kotlin, too, very soon.

Re: A little Golang way

#179

Post by swah » Thu, Aug 06, 2015, 11:14 AM UTC

jacquesm wrote: post #51

lpsz wrote: post #31

Anecdotes like this makes me wonder how much funding money and electricity could be saved if people migrated en masse from Ruby/Python/Something else to Go. (Edit: not meant to be a political statement, more of a practical observation. I only recently started using Go.)

That may be one of the bigger chances for go: mobile. After all being more efficient on mobile translates into longer battery life and go potentially has a huge advantage over the current java (ok, not java) based environment on Android. For google this would be a triple win, get out of the Oracle mess entirely, give their mobile developers a super fast toolchain and give the end users better battery life.

I love Golang, but its ain't very expressive, so it cannot be a good match for writing UIs. And Swift is much closer to Rust than to Go, IIUC...

Re: A little Golang way

#180

Post by anoother » Thu, Aug 06, 2015, 11:20 AM UTC

ambrice wrote: post #141

anoother wrote: post #132

Something troubles me about this article. I hope I'm misunderstanding it. It's stems from the following quotes; knowing this information, why do AeroFS still think Docker is a good fit for their use-case? > However, after our move to Docker, we noticed a sharp increase of the appliance's memory footprint. > ... > We identified several major factors behind these symptoms: > 1. an increase in the number of running JVMs…

Presumably Docker offers some benefit. It's weird that you and several others have jumped to the conclusion that it's obvious they should get rid of Docker. I would say: knowing this information, why do you still think java is a good fit for their use-case?

Because:

a) Their service was already written in it b) Their service was already performing adequately before they brought docker into play

But those aren't answers to your question. Java may have been a terrible fit for them, but their decision to move away from it was not motivated by that -- it was motivated by their already-written, already-performing Java code no longer performing as well, because of Docker.

The only question is, does Docker bring them enough benefits to justify reimplementing large amounts code, in a new and unfamiliar language? Maybe it does, but somehow I doubt it.