Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Go executable files are still getting larger

cockroachlabs.com

Post reply on HN
Print view View original

71–80 of 174 posts

Re: My Go executable files are still getting larger

#71

Post by whateveracct »

blinkingled wrote: post #30
parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?

Feels like the Go team could just...expose a flag?

Which they are loathe to do

Re: My Go executable files are still getting larger

#72

Post by whateveracct »

marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

It's the flip side to never exposing configuration flags and instead always making a Worse is Better decision for everyone.

It mirrors their GC philosophy.

Re: My Go executable files are still getting larger

#73

Post by rsc »

knz42 wrote: post #67
rsc wrote: post #65

Earlier quoted context omitted.

> For all we know, the go linker could be embedding random data. I do know, and it is not.

> But still, the sum of all the bytes advertised in symbol tables for the non-DWARF data does not sum up to the stripped size. What's the remainder about? if you do know, then pray, what is the answer to this question?

Maybe another day I will take the time to write a full-length blog post examining the bytes in a Go binary. Today I have other work planned and still intend to do it.

My points today are only that:

1. Go binary size has not gotten dramatically better or worse in any particular release and is mostly unchanged since Go 1.3.

2. Many claimed facts in the blog post are incorrect.

3. The linker is not "embedding random data" into Go binaries as you conjectured in the comment above.

Stepping back a level, you don't seem to be interested in engaging in good faith at all. I'm not going to reply to any more of your comments.

Re: My Go executable files are still getting larger

#74

Post by tediousdemise »

It seems to be a challenge to add zero-overhead features to programming languages.

Poor design decisions result in a language that gets extremely bloated over time, forcing you to use features that you don’t want to.

The better approach is to make these features optional, such as through a standard library.

Re: My Go executable files are still getting larger

#75

Post by erincandescent »

knz42 wrote: post #62
rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

As explained in another comment below, it's a good thing when all the byte usage is represented in the ELF section headers or the symbol tables. The DWARF data is currently so represented, and so was pclntab prior to 1.16. Today, the DWARF data is still present; the symbol for pclntab has an advertized size of zero, and the amount of data not represented in the symbol table and ELF section headers has grown. > If the…

> As explained in another comment below, it's a good thing when all the byte usage is represented in the ELF section headers or the symbol tables.

It's an ELF binary; all that's relevant are the program/segment headers and dynamic table. The section headers and non-dynamic symbol table are just there for orthogonality and to make the debugger's life a little easier (and the debugger would much prefer DWARF data and doesn't care about the section headers, tbh)

Re: My Go executable files are still getting larger

#76

Post by ClumsyPilot »

spockz wrote: post #33
parhamn wrote: post #21

Earlier quoted context omitted.

The article says they don't really care about initialization time though, which is right. Remember: cockroachdb is always synchronizing data across the cluster, that 175mb of ingress to start up a DB node, probably pales in comparison to data synchronization/relocations that happen on a cluster. Which is why worrying about ingress/egress costs over binary size is nonsense here too. The bandwidth you need to run a dis…

For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc . In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.

Here is a question: imagine you could double performance of cockroa hDB by making the executable 2000MB - every db admin would make that choice

Re: My Go executable files are still getting larger

#77

Post by majjgepolja »

whateveracct wrote: post #71
blinkingled wrote: post #30

Earlier quoted context omitted.

So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?

Feels like the Go team could just...expose a flag? Which they are loathe to do

Flags are juvenile. When I was young I used to play with flags. Now I set environment variables.

Re: My Go executable files are still getting larger

#79

Post by knz42 »

ClumsyPilot wrote: post #76
spockz wrote: post #33

Earlier quoted context omitted.

For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc . In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.

Here is a question: imagine you could double performance of cockroa hDB by making the executable 2000MB - every db admin would make that choice

This is effectively what is happening btw; the crdb binary went from 80MB to 200MB in the same time it took to make it twice as fast. The % growth in size is not a problem on its own; it's more the % size attributed to the program vs. the % size attributed to unclear purposes, that's a problem.

Re: My Go executable files are still getting larger

#80

Post by arp242 »

bradfitz wrote: post #64

To promote my own tool, https://github.com/bradfitz/shotizam lets you drill down into why Go binaries are large without having to make up terms like "dark bytes".

If physicists can have dark matter and dark energy, then why can't we have dark bytes? Why should we let the physicists have all the dark fun?
Post reply on HN