Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Go executable files are still getting larger

cockroachlabs.com

Post reply on HN
Print view View original

21–30 of 174 posts

Re: My Go executable files are still getting larger

#21

Post by parhamn »

gtirloni wrote: post #13
parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

It affects the download and instantiation time for containers.

The article says they don't really care about initialization time though, which is right.

Remember: cockroachdb is always synchronizing data across the cluster, that 175mb of ingress to start up a DB node, probably pales in comparison to data synchronization/relocations that happen on a cluster. Which is why worrying about ingress/egress costs over binary size is nonsense here too.

The bandwidth you need to run a distributed database cluster could download 172mb binary in milliseconds. If your node initiation time for DB failovers needs anything faster, you're doing something wrong.

There are stakeholders to this problem, Cockroach probably isn't one.

Re: My Go executable files are still getting larger

#22

Post by remus »

marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

> I always wonder if this is the flip side of the fast compilation?

I don't think there's an easy either-or answer to questions like this, but broadly language design seems to be about finding a sweet spot while balancing lots of trade offs, so to get fast compiles you're going to make trade-offs in other areas of the language and implementation. I imagine if the compilation time budget was higher there'd be some more space for binary pruning passes.

Re: My Go executable files are still getting larger

#24

Post by Touche »

parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

Go binary size makes it not an option for wasm.

Re: My Go executable files are still getting larger

#25

Post by rplnt »

marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

Fast compilation is great for development and testing. Both local and in CI.

Re: My Go executable files are still getting larger

#26

Post by agumonkey »

pjmlp wrote: post #12
londons_explore wrote: post #4

It's time for debug info like this to be sent to "onlinesymbolserver.com", encrypted with a hash of the binary. Then, whenever a debugger connects to a binary, it can simply download the symbols as required. And for the 99.9% who don't need debug info, it isn't needlessly shipped. Microsoft invented this in the 90's...

Around 1992 Borland C++ would use macro based code generation for generic code in BIDS 1.0, sounds familiar? I guess it is just a tradition with Go, rediscovering history.

On the memory capacity of social structures.pdf

Re: My Go executable files are still getting larger

#27

Post by gen220 »

Touche wrote: post #24
parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

Go binary size makes it not an option for wasm.

There are go variants for this. See tinygo [1], which targeted embedded originally (iirc) but now also targets wasm.

So you’re definitely correct that core Go is not an option, but options exist within the “greater metropolitan area” that’s built up around downtown.

These are among the benefits of having a relatively simple language and a 1.0 compatibility commitment, I think.

[1]: https://tinygo.org/ their FAQ is quite good.

Re: My Go executable files are still getting larger

#28

Post by [deleted] »

Touche wrote: post #24
parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

Go binary size makes it not an option for wasm.

[deleted]

Re: My Go executable files are still getting larger

#29

Post by blinkingled »

> We can call this the “dark file usage” of Go binaries, and it occupies between 15% and 33% of the total file size inside CockroachDB.

> Sadly, the removal of pclntab in Go 1.16 actually transferred the payload to the “dark” bytes.

I surely would have expect better from programming language designers/developers than this. Sounds like they just moved the problem from one place to another.

Re: My Go executable files are still getting larger

#30

Post by blinkingled »

parhamn wrote: post #9

> In other words, the Go team decided to make executable files larger to save up on initialization time. I mean... Im genuinely curious if this is a "we have extra engineering resources and can explore/complain about this" or "we have a client who is running cockroachdb and can't handle a 172mb binary install for a database server". Is there really someone out there who installs Cockroach (a global distributed auto-s…

So tomorrow if the Go team doubled your binary size by adding "darkest bytes" area with garbage in it your response would be to question the users's inability to be ok with it and not the Go team's choice and reasoning?
Post reply on HN