My Go executable files are still getting larger • Hacker News Board

Re: My Go executable files are still getting larger

#111

Post by knz42 » Fri, Apr 16, 2021, 5:07 PM UTC

mort96 wrote: post #105

knz42 wrote: post #84

Earlier quoted context omitted.

> The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint [...] yes this is acknowledged in the OP > We (the Go team) did not “recompress” pclntab in Go 1.15. There's now less redundancy in the funcdata, so in my book less redundancy = more compression. > We did not remove pclntab in Go 1.16. Correct; it is not "removed"; instead the advertised size of the symbol has been re…

The term "dark silicon" refers to silicon that's not (currently) in use, such as when you have a workload which only exercises one arithmetic unit of one core even if you have 8 cores with 4 arithmetic units each (only one arithmetic unit is "lit up", 63 are "dark"). There's no reason to believe that what you're calling "dark bytes" isn't actively used while executing the program.

Dark silicon is not used all the time - that's the key point.

In the same way, the functab data in Go is not used all the time either, only when generating stack traces.

Also since that original article from 2011 was published, the phrase "dark silicon" was extended to designate silicon IP which is not directly necessary for a given target application but is embedded for the occasional use by a niche in the same market.

In the same way, all languages (not just Go) embed constructs in generated code that are there only to handle edge cases in certain applications/deployments, and are inactive for 90%+ of executions.

Re: My Go executable files are still getting larger

#112

Post by saati » Fri, Apr 16, 2021, 5:07 PM UTC

arp242 wrote: post #80

bradfitz wrote: post #64

To promote my own tool, https://github.com/bradfitz/shotizam lets you drill down into why Go binaries are large without having to make up terms like "dark bytes".

If physicists can have dark matter and dark energy, then why can't we have dark bytes? Why should we let the physicists have all the dark fun?

Because it's genuinely unknown what those are, figuring out what these bytes are is just a deep dive into the opensource go compiler at worst or just asking around on some go channels for much less effort.

Re: My Go executable files are still getting larger

#113

Post by parhamn » Fri, Apr 16, 2021, 5:08 PM UTC

dvfjsdhgfv wrote: post #56

parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

> Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? I think it is exactly this mindset that caused the current situation (that the 2/3 of the compiled binaries are useless to the users).

The alternative isn't avoiding fixing it. The alternative is a boring bug report on github with actual stakeholders discussing the best strategies and trade offs.

This is only on HN's homepage because of the langauge flame wars. It's a garbage post.

Re: My Go executable files are still getting larger

#114

Post by the_mitsuhiko » Fri, Apr 16, 2021, 5:13 PM UTC

icholy wrote: post #109

the_mitsuhiko wrote: post #85

Earlier quoted context omitted.

Not for unwinding or line programs.

What's your point exactly?

That go is running their own stuff instead of embracing existing standards. Symbol servers and split dwarf thus is not likely to be supported.

Re: My Go executable files are still getting larger

#115

Post by knz42 » Fri, Apr 16, 2021, 5:15 PM UTC

mort96 wrote: post #110

knz42 wrote: post #104

Earlier quoted context omitted.

> Go executable sizes haven't increased much in general. Russ's example was just the "gofmt" program. > Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing more code for Cockroach DB? If that was the only reason, then the % overhead would remain constant-ish. But it is increasing. So there is a non-linear factor for _some_ go programs (like cockroachdb) and it's still unclear what t…

It's not clear that the overhead is due to Go itself producing bigger binaries over time though. If you recompiled all the different CockroachDB versions with Go 1.8 (if that was feasible), it's quite probable that the tables you would end up with would look fairly similar to the ones you're actually showing. If there is superlinear growth in binary sizes as the project grows – for example, if some part is O(n^2) in…

> tried to blame it partly on the Go compiler producing more bloated code over time

Where? The argument is _precisely_ that the growth is occurring in non-code areas.

> partly on a mystical "dark area" which you don't understand

The _observation_ is that the growth is happening in an area of the file that's not accounted for in the symtable. That's what makes it "dark". It's not mythical: it's _there_ and you can observe it just as well as anyone else.

> you mentioned superlinear growth only in the comment section

it's in the reported measurements.

> and you didn't actually gather data or do experiments to prove or disprove any of the things you're claiming as a cause

The analysis is stating observations and reporting that the size is increasingly due to non-accounted data. That observation is substantiated by measurements. There's no claim of cause in the text!

Re: My Go executable files are still getting larger

#116

Post by rsc » Fri, Apr 16, 2021, 5:16 PM UTC

rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

Another thing I noticed in the revised blog post on a second skim, regarding this claim:

> Every time, 70% of a couple hundred megabytes are copied around for no good reason and someone needs to pay ingress/egress networking costs for these file copies.

That 70% includes the ELF/DWARF metadata that is easily removed from the binary using strip. It's true that the DWARF info in particular has gotten larger in recent releases, because we've included more information to make debuggers work better. I don't think it has grown quite as rapidly as the table indicates - I think some of the rows already have some of the debug metadata removed in "Raw size".

Regardless, I would hope that anyone sensitive to networking costs at this level would be shipping around stripped binaries, so the growth in accurate DWARF info should not be relevant to this post at all.

That is, the right comparison is to the "Stripped" column in the big table. If you subtract out the strippable overheads and you take the "Dark + pclntab" as an accurate representation of Go-specific overhead (debatable but not today), then the situation has actually improved markedly since Go 1.12, which would have been current in April 2019 when the first post was written.

Whereas in Go 1.12 the measured "actual program" was only about 40% of the stripped binary, in Go 1.16 that fraction has risen to closer to 55%.

This is a marked-up copy of the table from the dr-knz.net revised post that at time of writing has not yet made it to cockroachlabs.com: https://swtch.com/tmp/cockroach-blog.png

I think the numbers in the table may be suspect in other ways, so I am NOT claiming that from Go 1.12 to Go 1.16 there has actually been a 15% reduction in "Go metadata overhead". I honestly don't know one way or the other without spending a lot more time looking into this.

But supposing we accept for sake of argument that the byte counts in the table are valid, they do not support the text or the title of the post. In fact, they tell the opposite story: the stripped CockroachDB binary in question has gotten smaller since April 2019, and less of the binary is occupied by what the post calls "non-useful" or "Go-internal" bytes.

Re: My Go executable files are still getting larger

#117

Post by Scarbutt » Fri, Apr 16, 2021, 5:16 PM UTC

knz42 wrote: post #104

returningfory2 wrote: post #99

Earlier quoted context omitted.

In this case, what is the point the blog post is trying to make? The title of the post is "Go Executable Files Are Still Getting Larger". Upon further reading and conversation here it seems this is possibly not true, nor what the post is about. If we believe Russ's comments, Go executable sizes haven't increased much in general. Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing mo…

> Go executable sizes haven't increased much in general. Russ's example was just the "gofmt" program. > Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing more code for Cockroach DB? If that was the only reason, then the % overhead would remain constant-ish. But it is increasing. So there is a non-linear factor for _some_ go programs (like cockroachdb) and it's still unclear what t…

CockroachDB has always had a reputation of being slow, 10x-20x slower than the same operation being made in Postgres, with this and the issues about binary size, was a GC language like Go the right choice for CK? Would you have pick something else today if starting a new?

Re: My Go executable files are still getting larger

#118

Post by arp242 » Fri, Apr 16, 2021, 5:22 PM UTC

knz42 wrote: post #108

arp242 wrote: post #98

Earlier quoted context omitted.

> Look you can claim that "having the necessary metadata for garbage collection and reflection in a statically-compiled language takes up a significant amount of space" but without clear evidence of how much space that is, with proper accounting of byte usage, this claim is non-falsifiable and thus of doubtful value. The article made the claim that 70% of space is wasted "dark bytes". The article should prove the cla…

> The article made the claim that 70% of space is wasted "dark bytes" This is incorrect. The claim is that the bytes are either non-accounted, or motivated by technical choices specific to Go. > What you're doing here is pretty much the same trick quacks [...] Look the article has some measurements with numbers which you can readily reproduce on your own computer, and the methodology is even described. The main claim…

> The claim is that the bytes are either non-accounted, or motivated by technical choices specific to Go.

That's not what it says; with "70% of a couple hundred megabytes are copied around for no good reason" written in bold no less:

> That’s right! More than two thirds of the file on disk is taken by… bits of dubious value to the software product. > > Moreover, consider that these executable files fly around as container images, and/or are copied between VMs in the cloud, thousands of times per day! Every time, 70% of a couple hundred megabytes are copied around for no good reason and someone needs to pay ingress/egress networking costs for these file copies. That is quite some money being burned for no good reason!

Re: My Go executable files are still getting larger

#119

Post by arp242 » Fri, Apr 16, 2021, 5:23 PM UTC

saati wrote: post #112

arp242 wrote: post #80

Earlier quoted context omitted.

If physicists can have dark matter and dark energy, then why can't we have dark bytes? Why should we let the physicists have all the dark fun?

Because it's genuinely unknown what those are, figuring out what these bytes are is just a deep dive into the opensource go compiler at worst or just asking around on some go channels for much less effort.

You're taking a joke too serious :-)

Re: My Go executable files are still getting larger

#120

Post by knz42 » Fri, Apr 16, 2021, 5:24 PM UTC

rsc wrote: post #116

rsc wrote: post #55

This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…

Another thing I noticed in the revised blog post on a second skim, regarding this claim: > Every time, 70% of a couple hundred megabytes are copied around for no good reason and someone needs to pay ingress/egress networking costs for these file copies. That 70% includes the ELF/DWARF metadata that is easily removed from the binary using strip. It's true that the DWARF info in particular has gotten larger in recent r…

Thanks Russ for that additional insight.

> I would hope that anyone sensitive to networking costs at this level would be shipping around stripped binaries, so the growth in accurate DWARF info should not be relevant to this post at all.

Good point. I removed that part from the conclusion.

> If you subtract out the strippable overheads and you take the "Dark + pclntab" as an accurate representation of Go-specific overhead [...] then the situation has actually improved markedly since Go 1.12 [...] Whereas in Go 1.12 the measured "actual program" was only about 40% of the stripped binary, in Go 1.16 that fraction has risen to closer to 55%.

Ok, that is fair. I will attempt to produce a new version of these tables with this clarification.

> the stripped CockroachDB binary in question has gotten smaller since April 2019, and less of the binary is occupied by what the post calls "non-useful" or "Go-internal" bytes.

There's an explanation for that, which is that the crdb code was also reduced in that time frame.