101–110 of 174 posts
> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…
> Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? I think it is exactly this mindset that caused the current situation (that the 2/3 of the compiled binaries are useless to the users).
That the bytes are not visible in the symbol table is inarguable, that they are useless is a highly contentious statement and very probably wrong.
Python is still more wildly used / popular language, but I never seen a Python container for a real project which was less than 1GB.
Earlier quoted context omitted.
> The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint [...] yes this is acknowledged in the OP > We (the Go team) did not “recompress” pclntab in Go 1.15. There's now less redundancy in the funcdata, so in my book less redundancy = more compression. > We did not remove pclntab in Go 1.16. Correct; it is not "removed"; instead the advertised size of the symbol has been re…
In this case, what is the point the blog post is trying to make? The title of the post is "Go Executable Files Are Still Getting Larger". Upon further reading and conversation here it seems this is possibly not true, nor what the post is about. If we believe Russ's comments, Go executable sizes haven't increased much in general. Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing mo…
Russ's example was just the "gofmt" program.
> Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing more code for Cockroach DB?
If that was the only reason, then the % overhead would remain constant-ish. But it is increasing. So there is a non-linear factor for _some_ go programs (like cockroachdb) and it's still unclear what that factor is.
This article is full of misinformation. Just a few representative things: - The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint, by letting the OS demand-page this critical table that is used any time a stack must be walked (in particular, during garbage collection). See https://golang.org/s/go12symtab for details. - We (the Go team) did not “recompress” pclntab in Go 1.…
> The expansion of pclntab in Go 1.2 dramatically improved startup time and reduced memory footprint [...] yes this is acknowledged in the OP > We (the Go team) did not “recompress” pclntab in Go 1.15. There's now less redundancy in the funcdata, so in my book less redundancy = more compression. > We did not remove pclntab in Go 1.16. Correct; it is not "removed"; instead the advertised size of the symbol has been re…
> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…
So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?
Earlier quoted context omitted.
> Is there really someone out there who installs Cockroach (a global distributed auto-sharded database) and thinks twice about 172mb of disk space? I think it is exactly this mindset that caused the current situation (that the 2/3 of the compiled binaries are useless to the users).
> (that the 2/3 of the compiled binaries are useless to the users). That the bytes are not visible in the symbol table is inarguable, that they are useless is a highly contentious statement and very probably wrong.
Earlier quoted context omitted.
As explained in another comment below, it's a good thing when all the byte usage is represented in the ELF section headers or the symbol tables. The DWARF data is currently so represented, and so was pclntab prior to 1.16. Today, the DWARF data is still present; the symbol for pclntab has an advertized size of zero, and the amount of data not represented in the symbol table and ELF section headers has grown. > If the…
> Look you can claim that "having the necessary metadata for garbage collection and reflection in a statically-compiled language takes up a significant amount of space" but without clear evidence of how much space that is, with proper accounting of byte usage, this claim is non-falsifiable and thus of doubtful value. The article made the claim that 70% of space is wasted "dark bytes". The article should prove the cla…
This is incorrect. The claim is that the bytes are either non-accounted, or motivated by technical choices specific to Go.
> What you're doing here is pretty much the same trick quacks [...]
Look the article has some measurements with numbers which you can readily reproduce on your own computer, and the methodology is even described. The main claim is that "it's unclear what these bytes are about". The previous claim that they were "non-useful" was retracted. The data is there, and there's a question: "What is this data about?"
The text is even doubling down by spelling out "there's no satisfying explanation yet".
> outright disbelieving people like this is rather rude
We're not in the business of "believing" or "disbelieving" here I think? There's data, there's measurements, and there are explanations.
After my comments and that of others, Russ provided a more elaborate, more detailed (and at last, falsifiable in the positive, epistemological sense of the word) explanation deeper in the thread. Now we can make the work of looking into it and check the explanation.
> your comment has the strong implication that Russ is outright lying
Your understanding is flawed then? There was no assumption of lies implied.
Earlier quoted context omitted.
In this case, what is the point the blog post is trying to make? The title of the post is "Go Executable Files Are Still Getting Larger". Upon further reading and conversation here it seems this is possibly not true, nor what the post is about. If we believe Russ's comments, Go executable sizes haven't increased much in general. Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing mo…
> Go executable sizes haven't increased much in general. Russ's example was just the "gofmt" program. > Perhaps the reason you're seeing increases in Cockroach DB is because you keep writing more code for Cockroach DB? If that was the only reason, then the % overhead would remain constant-ish. But it is increasing. So there is a non-linear factor for _some_ go programs (like cockroachdb) and it's still unclear what t…
If there is superlinear growth in binary sizes as the project grows – for example, if some part is O(n^2) in the number of interfaces – then that's certainly interesting. If you demonstrated that such superlinear growth is happening, and wrote an article based on that, people wouldn't be so critical.
If Go binaries are getting bigger because Go produces bigger binaries for the same source code over time, then that's also interesting. If you demonstrated that Go binaries are getting more and more bloated over time for the same source code, and wrote an article based on that, people wouldn't be critical.
But as it is, you kind of just complained that CockroachDB is getting bigger, tried to blame it partly on the Go compiler producing more bloated code over time, partly on a mystical "dark area" which you don't understand, you mentioned superlinear growth only in the comment section, and you didn't actually gather data or do experiments to prove or disprove any of the things you're claiming as a cause. That's why people are complaining.