Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Go executable files are still getting larger

cockroachlabs.com

Post reply on HN
Print view View original

161–170 of 174 posts

Re: My Go executable files are still getting larger

#161

Post by sagichmal »

oefrha wrote: post #43
gen220 wrote: post #27

Earlier quoted context omitted.

There are go variants for this. See tinygo [1], which targeted embedded originally (iirc) but now also targets wasm. So you’re definitely correct that core Go is not an option, but options exist within the “greater metropolitan area” that’s built up around downtown. These are among the benefits of having a relatively simple language and a 1.0 compatibility commitment, I think. [1]: https://tinygo.org/ their FAQ is qu…

Tinygo is severely limited, e.g. it doesn't support encoding/json. Want to deal with any JSON API, or anything that indirectly uses JSON serialization? Forget about it. A current side project of mine uses golang's wasm. Not a big codebase, but the wasm is 2.7MB brotli'ed. Certainly huge to me (I'm sure it's almost in the lean and mean camp compared to the average website today, though).

TinyGo will soon support encoding/json.

Re: My Go executable files are still getting larger

#162

Post by gampolt »

rsc wrote: post #61
rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

An easily obtained apples-to-apples¹ table: $ for i in $(seq 3 16); do curl -sLo go1.$i.tgz https://golang.org/dl/go1.$i.linux-amd64.tar.gz tar xzf go1.$i.tgz go/bin/gofmt size=$(ls -l go/bin/gofmt | awk '{print $5}') strip go/bin/gofmt size2=$(ls -l go/bin/gofmt | awk '{print $5}') echo go1.$i $size $size2 done go1.3 3496520 2528664 go1.4² 14398336 13139184 go1.5 3937888 2765696 go1.6 3894568 2725376 go1.7 3036195 1…

https://lobste.rs/s/gvtstv/my_go_executable_files_are_still_... here is apple to apple compare for compile cockroachDB 1.0 across go 1.8 to 1.16, it's 20% smaller not bigger go 1.8 = 58,099,688 go 1.16 = 47,317,624

Re: My Go executable files are still getting larger

#163

Post by rsc »

rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

The article has been updated to avoid calling the bytes "not useful" and to address the narrow, specific points I raised, but it is still generally suspect.

Over on Lobsters, zeebo took the time (thanks!) to build the same version of CockroachDB with various historical Go toolchains, varying only the toolchain, and found that if anything the Go executables are getting smaller over time, in some cases significantly so.

            v1.0          v20.2.0
    1.8     58,099,688    n/a
    1.9     57,897,616    314,191,032
    1.10    57,722,520    313,669,616
    1.11    48,961,712    233,170,304
    1.12    52,440,168    236,192,600
    1.13    50,844,048    214,373,144
    1.14    50,527,320    212,699,656
    1.15    47,910,360    201,391,416
https://lobste.rs/s/gvtstv/my_go_executable_files_are_still_...

The title of the article ("My Go executable files are still getting larger") appears to be literally untrue, at least read as a critique of Go itself. If they are getting larger, it's because new code is being added, not because the Go runtime or compiler is degrading in some way over time.

Re: My Go executable files are still getting larger

#164

Post by knz42 »

rsc wrote: post #163
rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

The article has been updated to avoid calling the bytes "not useful" and to address the narrow, specific points I raised, but it is still generally suspect. Over on Lobsters, zeebo took the time (thanks!) to build the same version of CockroachDB with various historical Go toolchains, varying only the toolchain, and found that if anything the Go executables are getting smaller over time, in some cases significantly so…

> The title of the article ("My Go executable files are still getting larger") appears to be literally untrue, at least read as a critique of Go itself. If they are getting larger, it's because new code is being added, not because the Go runtime or compiler is degrading in some way over time.

Yes this is a fair assessment, although I find it surprising (and enlightening) that you refer to “a critique of Go”. At no moment was the intent to critique Go specifically; the entire analysis is made of observation of the results of combining Go with specific (and varying) amounts of source code.

In any case, based on this discussion I have decided to amend the title and emphasize in the conclusion that the absolute size of the code+data for a fixed amount of source code has decreased between go 1.15 and 1.16.

edit: This is relevant to this discussion: https://sneak.berlin/20191201/american-communication/

Re: My Go executable files are still getting larger

#166

Post by anthk »

eklitzke wrote: post #156
anthk wrote: post #58

Earlier quoted context omitted.

Go's static linking idea comes from plan9 C compilers, a few years before Java. We owe a lot from plan9: - Go's design, based on both C compilers and Inferno's limbo - /proc - utf-8 - 9p

/proc does not come from plan9: http://dtrace.org/blogs/eschrock/2004/06/25/a-brief-history-...

Unix 8/10 -> Plan9 :).

Rio's foundations come from the Blit.

Unix 8/10 was not a big success, but a lot of ideas from that went into Plan9.

Re: My Go executable files are still getting larger

#167

Post by pjmlp »

anthk wrote: post #138
synergy20 wrote: post #70

Earlier quoted context omitted.

in that case you can happily static-link c++ programs, which probably will be smaller than go binary.

happily and "statically link c++" don't match together well.

They do, there are more C++ compilers out there than gcc with glibc.

Re: My Go executable files are still getting larger

#168

Post by remram »

hatuio wrote: post #165
remram wrote: post #157

Earlier quoted context omitted.

So you did mean partition/file, and you are misinformed.

So it is getting loaded as file and can be swapped out due to this?

Yes, any mmap'd file will be swapped in on demand, when a page is first accessed, it will not all be copied in physical memory at once. In case of memory pressure, pages will be removed from physical memory ("swapped out"), since they can be loaded back from the file again when needed.

Since the executable is mapped read-only, the pages loaded in physical memory can also be shared between multiple instances of the process.

Re: My Go executable files are still getting larger

#169

Post by spockz »

ClumsyPilot wrote: post #76
spockz wrote: post #33

Earlier quoted context omitted.

For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc . In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.

Here is a question: imagine you could double performance of cockroa hDB by making the executable 2000MB - every db admin would make that choice

Yes, if it would double the performance in many cassis, then the size would have a significant benefit.

Here we are taking about large binaries without apparent benefit and a drive to keep binaries as lean as possible.

Re: My Go executable files are still getting larger

#170

Post by rsc »

knz42 wrote: post #164
rsc wrote: post #163

Earlier quoted context omitted.

The article has been updated to avoid calling the bytes "not useful" and to address the narrow, specific points I raised, but it is still generally suspect. Over on Lobsters, zeebo took the time (thanks!) to build the same version of CockroachDB with various historical Go toolchains, varying only the toolchain, and found that if anything the Go executables are getting smaller over time, in some cases significantly so…

> The title of the article ("My Go executable files are still getting larger") appears to be literally untrue, at least read as a critique of Go itself. If they are getting larger, it's because new code is being added, not because the Go runtime or compiler is degrading in some way over time. Yes this is a fair assessment, although I find it surprising (and enlightening) that you refer to “a critique of Go”. At no mo…

In addition to the title (why mention "Go" if it was not a critique of Go?), you also wrote in bold:

- "These Go executable files are rather... bloated." - "70% of a couple hundred megabytes are copied around for no good reason"

I find it hard to believe that even a European software team would not consider those direct criticisms.

Post reply on HN