Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

My Go executable files are still getting larger

cockroachlabs.com

Post reply on HN
Print view View original

31–40 of 174 posts

Re: My Go executable files are still getting larger

#31

Post by bbatha »

stabbles wrote: post #15

> Every time, 70% of a couple hundred megabytes are copied around for no good reason and someone needs to pay ingress/egress networking costs for these file copies. That is quite some money being burned for no good reason! Meanwhile half of the world is pushing images by nvidia, intel and amd around for their machine learning software: Intel OneAPI runtime libraries: 4.74GB (or 18.4GB for compilers) CUDA runtime libr…

Most of the time, especially with the docker hub rate limiting changes people are or should be using their cloud provider's mirror or running their own mirror. Actually ingressing the docker image into your network should happen once in most production setups.

Re: My Go executable files are still getting larger

#32

Post by tedunangst »

> Every time, 70% of a couple hundred megabytes are copied around for no good reason and someone needs to pay ingress/egress networking costs for these file copies.

Just zero them out and they'll compress to nothing. Even better, with a sparse file aware tool like tar, they won't even use disk space.

Re: My Go executable files are still getting larger

#33

Post by spockz »

parhamn wrote: post #21
gtirloni wrote: post #13

Earlier quoted context omitted.

It affects the download and instantiation time for containers.

The article says they don't really care about initialization time though, which is right. Remember: cockroachdb is always synchronizing data across the cluster, that 175mb of ingress to start up a DB node, probably pales in comparison to data synchronization/relocations that happen on a cluster. Which is why worrying about ingress/egress costs over binary size is nonsense here too. The bandwidth you need to run a dis…

For production, yes. But it also effects startup and download time on developer machines. Want to have multiple versions installed? Now it takes more space. Takes longer to download on 4g while on the road or on crappy corporate/conference WiFi, etc .

In the end this all ends up because it is for all go binaries. I’ve come appreciate attention for leanness because in the end it does add up.

Re: My Go executable files are still getting larger

#36

Post by cpuguy83 »

throwaway894345 wrote: post #14
marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

> I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. Presumably a few mb of disk usage also doesn't matter then? I also write web services, but I do care about the init time precisely because I want to be deployments to take as little time as possible so that we can deploy (and rollback) many times per day with relatively simple automation. Tha…

Unless you are running these in containers, then that time adds up (assuming you are using go-based container runtime implementations, as most people are).

Re: My Go executable files are still getting larger

#37

Post by marcus_holmes »

throwaway894345 wrote: post #14
marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

> I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. Presumably a few mb of disk usage also doesn't matter then? I also write web services, but I do care about the init time precisely because I want to be deployments to take as little time as possible so that we can deploy (and rollback) many times per day with relatively simple automation. Tha…

Yeah, it's a trade-off. At the moment the longest part of my deploy is copying the new executable to the server. I'd trade a couple of seconds of init time for a smaller executable because that would result in a faster deploy.

I don't use Docker to deploy, because it's just a single executable file (and a bunch of templates, though I'm looking at embedding those). One of the reasons I'm reluctant to go down the Docker road is because it's going to add more time to my deployment.

Re: My Go executable files are still getting larger

#38

Post by marcus_holmes »

rplnt wrote: post #25
marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

Fast compilation is great for development and testing. Both local and in CI.

totally agree. Being able to hit the "run this test" and not having to wait for it to compile is awesome :)

But if there some flags to tell the compiler to compile slowly and produce a smaller executable, that would be awesome for final builds that need to be shipped across the internet.

Re: My Go executable files are still getting larger

#39

Post by skrebbel »

daitangio wrote: post #35

Go static linking is a great happy idea for a Java guy trapped in the Classpath Dependency Hell (or C# / DLL Hell). It is a very annoying thing for a C++ programmer, which can dynamically link operating system libraries at will.

You mean a C++ programmer who does not care whether their program works on anybody else's computer?

Re: My Go executable files are still getting larger

#40

Post by kortex »

marcus_holmes wrote: post #3

I always wonder if this is the flip side of the fast compilation? It would be nice to be able to decide on those trade-offs ourselves. I mostly write web servers in Go, which (as the article says) are executed rarely, so init time really doesn't matter to me. But I've been looking at writing some desktop apps in Go, and then init time will matter.

At least for CLI-tool-sorta-programs, I find Rust and Go emit vaguely similar sized binaries, a few Mb is typical. The biggest cause is of course static linking.

However, once you exceed the 10-20Mb regime, I seem to find more 20-200Mb Go bins in the wild. This can be misleading, since the Go world is a fan of packing assets into binaries, eg for web GUIs.

What dings Go bins on init time is the Go runtime. Still, it's rarely noticeable unless you are firing thousands of times a second.

Post reply on HN