A little Golang way • Hacker News Board

Re: A little Golang way

#131

Post by pcwalton » Wed, Aug 05, 2015, 9:51 PM UTC

dragonwriter wrote: post #125

cgag wrote: post #124

Earlier quoted context omitted.

I don't know if the programming model of threads and channels that's used by haskell and golang strictly depends on the existence of M:N threading, but it sure is nicer to use than the model in java/rust.

It doesn't strictly depend on it, but it is strongly supported by M:N threading since that threading model decouples the supportable degree of concurrency from the supportable degree of parallelism (unlike 1:1), without abandoning parallelism (unlike N:1). You can do Erlang-style concurrency with 1:1 or N:1 threading models (and there are libraries for languages whose many implementations are N:1 or 1:1 rather than M…

> It doesn't strictly depend on it, but it is strongly supported by M:N threading since that threading model decouples the supportable degree of concurrency from the supportable degree of parallelism (unlike 1:1), without abandoning parallelism (unlike N:1).

A modern Linux kernel has excellent thread scalability, and you can customize thread stack sizes to improve memory usage. (Thread memory use is kind of independent of 1:1 vs. M:N, honestly; the thing that can improve scalability is relocatable stacks, which is really not the same thing.)

It's instructive to look at the history of M:N versus 1:1 in the days of NPTL and LinuxThreads and compare that to the history of programming languages. In that world it was received wisdom that M:N would be superior for the reasons always cited today, but at the end of the day 1:1 was found to be better in practice, because the Linux kernel is pretty darn good at scheduling and pretty darn fast at syscalls. Nobody advocates M:N anymore in the Linux world; it's universally agreed to be a dead end. Now, to be fair to Golang, relocatable stacks do change the equation somewhat, but (a) as argued above, I think that's really independent of 1:1 vs. M:N; (b) any sort of userspace threading won't get you to the performance of, say, nginx, as once you have any sort of stack per "thread" you've already lost.

Re: A little Golang way

#132

Post by anoother » Wed, Aug 05, 2015, 9:57 PM UTC

Something troubles me about this article. I hope I'm misunderstanding it.

It's stems from the following quotes; knowing this information, why do AeroFS still think Docker is a good fit for their use-case?

> However, after our move to Docker, we noticed a sharp increase of the appliance's memory footprint.

> ...

> We identified several major factors behind these symptoms:

> 1. an increase in the number of running JVMs, as each tomcat servlet was placed into a separate container

> 2. reduced opportunity for the many JVMs to share read-only memory: the JVM itself, all the shared libraries it depends on, and of course the many JARs used by multiple services

> 3. memory isolation could in some cases confuse some sizing heuristics, which lead to larger caches being allocated by some services

Re: A little Golang way

#133

Post by djhworld » Wed, Aug 05, 2015, 10:01 PM UTC

I find this article confusing, in the first part the writer talks about Tomcat and servlets, in the second part he talks about reducing the LOC from 175 lines to 96 by using Go.

To me it seems the Java solution was wildly over engineered and probably could have been refactored to not even need Tomcat.

Re: A little Golang way

#134

Post by dragonwriter » Wed, Aug 05, 2015, 10:05 PM UTC

pcwalton wrote: post #131

dragonwriter wrote: post #125

Earlier quoted context omitted.

It doesn't strictly depend on it, but it is strongly supported by M:N threading since that threading model decouples the supportable degree of concurrency from the supportable degree of parallelism (unlike 1:1), without abandoning parallelism (unlike N:1). You can do Erlang-style concurrency with 1:1 or N:1 threading models (and there are libraries for languages whose many implementations are N:1 or 1:1 rather than M…

> It doesn't strictly depend on it, but it is strongly supported by M:N threading since that threading model decouples the supportable degree of concurrency from the supportable degree of parallelism (unlike 1:1), without abandoning parallelism (unlike N:1). A modern Linux kernel has excellent thread scalability, and you can customize thread stack sizes to improve memory usage. (Thread memory use is kind of independe…

> A modern Linux kernel has excellent thread scalability, and you can customize thread stack sizes to improve memory usage.

Cross-platform code can't count on always running on a modern Linux kernel, though.

But, sure, I'd think that M:N (which is never free) is going to be less likely to be worth the cost if you are specifically targeting an underlying platform reduces the cost of high numbers of native threads and the cost of native thread switching so that the price for user-level threading in the runtime isn't buying you improvements in those areas.

Re: A little Golang way

#135

Post by cgag » Wed, Aug 05, 2015, 10:09 PM UTC

pcwalton wrote: post #130

cgag wrote: post #124

Earlier quoted context omitted.

I don't know if the programming model of threads and channels that's used by haskell and golang strictly depends on the existence of M:N threading, but it sure is nicer to use than the model in java/rust.

Why is it nicer to use than that of Rust?

Well, if what you're saying about not needing M:N to get thread scalability is true, I'm probably wrong. I really like being able to run tons of threads and write synchronous code, letting the the haskell RTS swap threads when I block.

My mental model, which probably comes from everyone complaining about apaches thread-per-request model a few years ago, is that you basically either consume lots of resources with lots of threads, or you write ugly cps style code. I view haskell/go as giving the best of both worlds, but perhaps they aren't separate worlds after all.

Re: A little Golang way

#136

Post by icebraining » Wed, Aug 05, 2015, 10:23 PM UTC

tracker1 wrote: post #41

mkozlows wrote: post #24

Earlier quoted context omitted.

34MB for Hello World, according to this SO post: http://stackoverflow.com/questions/13692206/high-java-memory... I'm not going to install the JDK just to verify, but feel free to report back if you get different results.

For comparison... I did a similar Hello World test around the timeframe of .Net 2.0, and it was around 11mb to load IIRC. The latest iojs on windows seems to be just under 9mb. Not sure what the golang overhead is, by comparison.

I can't say what an Hello World uses, but I have a Tumblr API → RSS gateway written in Go running for a few months, and it uses ~2MB.

Re: A little Golang way

#137

Post by [deleted] » Wed, Aug 05, 2015, 11:04 PM UTC

flippant wrote: post #127

mavelikara wrote: post #121

I have a question on Go's performance that I'd like someone here, who has experience writing programs in it, to help shed some light upon. 1) Go does not have a runtime. This means that there is no JIT to do any optimizations based on runtime profiling. 2) Go is also designed to compile fast. Since it compiles fast, the compiler's time budget to do compile-time optimizations is small and it probably can't do the best…

1) Go has a runtime that's included in the binary that you get when you run `go build`. It takes care of garbage collection, goroutines etc. It does not do any JIT compilation afaik. 2) Go compiles fast because it gives up a lot of modern features (namely generics).

[deleted]

Re: A little Golang way

#138

Post by Scramblejams » Wed, Aug 05, 2015, 11:14 PM UTC

cgag wrote: post #135

pcwalton wrote: post #130

Earlier quoted context omitted.

Why is it nicer to use than that of Rust?

Well, if what you're saying about not needing M:N to get thread scalability is true, I'm probably wrong. I really like being able to run tons of threads and write synchronous code, letting the the haskell RTS swap threads when I block. My mental model, which probably comes from everyone complaining about apaches thread-per-request model a few years ago, is that you basically either consume lots of resources with lots…

I feel the same about Erlang processes. It's so much easier on my brain to write synchronous code that's preemptively executed and communicating by message passing. Wish Rust gave me that option, speed hit and all, but I understand why it doesn't.

Re: A little Golang way

#139

Post by pcwalton » Wed, Aug 05, 2015, 11:21 PM UTC

Scramblejams wrote: post #138

cgag wrote: post #135

Earlier quoted context omitted.

Well, if what you're saying about not needing M:N to get thread scalability is true, I'm probably wrong. I really like being able to run tons of threads and write synchronous code, letting the the haskell RTS swap threads when I block. My mental model, which probably comes from everyone complaining about apaches thread-per-request model a few years ago, is that you basically either consume lots of resources with lots…

I feel the same about Erlang processes. It's so much easier on my brain to write synchronous code that's preemptively executed and communicating by message passing. Wish Rust gave me that option, speed hit and all, but I understand why it doesn't.

How does Rust not give you that option?

The choice of 1:1 and M:N does not affect the programming model at all. It is simply an implementation detail. M:N scheduling is no more a requirement for CSP than Unix is for TCP.

Re: A little Golang way

#140

Post by vonsnowman » Wed, Aug 05, 2015, 11:29 PM UTC

kentonv wrote: post #79

> Resident memory usage dropped from 87MB down to a mere 3MB, a 29x reduction! This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons. With an interpreter or JIT, you need to load all your code at startup and process it. Generally, _all_ of your dependencies need to be loaded and parsed upfront, either converted to some intern…

> This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons. > [...] while an app written in C++, Rust, or Go will take 2MB. Agreed. As mentioned in the blog post, I considered Rust but decided against it because of I found it much less mature than Go. I did not consider C++ because, as mentioned in the blog post, part of the point was to experiment with new language/tools and even though I consider myself proficient with it, I learned the hard way that the lack of memory safety is rarely worth it.

> I suspect that the 668MB Java image was at least 90% unnecessary garbage that was not actually needed at runtime. Unfortunately the package managers we all use are not optimized for containers Exactly. That was part of the point of this blog post, which I may not have been successful at getting across. Switching to go was, if not the path of least resistance to solve this issue, at least one of a few relatively easy routes. It also happened to be a great deal of fun.