Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Static Linux

sta.li

Post reply on HN
Print view View original

31–40 of 61 posts

Re: Static Linux

#31

Post by zackmorris »

deathanatos wrote: post #28
zackmorris wrote: post #23

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? Also, a program A depends on library B and library C v1.1. Library B also depends on C, but v1.2. Which gets used? > More often than not, when…

Ya it would basically ignore names/versions and just use the hash as the identifier, a bit like BitTorrent.

Unfortunately you do bring up a good point that security patches generally won't work without a recompile of the parent binary. One possible way out of this is that the external interface/unit tests for a library could also have their own hash, so if a library fixes something like a buffer exploit without changing its interface, the parent binary could use the new drop-in replacement. In practice I’m skeptical if this would work reliably though, because binaries may be relying on idiosyncrasies.

I'm thinking that a simpler method would be to have patches use the hash system. So say curl uses libssl and libssl releases a security update, then someone could drop the new libssl into the curl project and rebuild it without having to touch any code, giving curl a new hash that could be installed by other users. I think we are used to this almost never working so we are hesitant to upgrade. But the idea would be that we’d upgrade binaries that depend on libraries (rather than just libraries) and it would be a really cheap operation compared to today.

Re: Static Linux

#32

Post by GeorgeTirebiter »

Lerc wrote: post #20
ultramancool wrote: post #17

Cool idea. Not enough people seem aware of executable packers like UPX http://upx.sourceforge.net/ though. These are excellent tools to keep the size down when using large static binaries. By compressing the file on disk and decompressing in memory you often wind up with a smaller and sometimes even faster loading (depending on disk IO speed vs decompression speed) package. I got a static Qt binary from 4 MB down to…

The downside to tools like UPX is that the executable code is actively transformed on load. This limits the ability to use shared memory for multiple concurrent executions of the same executable. If the OS loads executables by mmap and load on page-hit, you can potentially save memory by not ever loading unused parts of an executable. a transform-on-load requires the entire program to be loaded before execution begin…

'... never loading unused parts of an executable...' - this is one of the benefits of a paging system. Paging was (is?) the best way to keep memory requirement down. The way Multics worked was by having paged segments. http://www.osinfoblog.com/post/136/segmentation-with-paging:... The x86(-64) can support something like this, but as far as I know, no modern OS supports this feature.

Re: Static Linux

#33

Post by icebraining »

Animats wrote: post #30
leakybucket wrote: post #26

An advantage of dynamic libraries is that the memory used to hold the library's executable pages can be shared across processes. So using static only binaries will lead to less free memory on the OS.

That's the party line. It's often wrong. If two copies of the same program are running, they share memory for code. For a shared library to reduce memory consumption, there must be multiple different programs using the same version of the same library. That's not all that common, beyond very basic libraries such as "libc". Linking to a shared library brings in and initializes the whole library, even if you only need…

That's not all that common

Isn't it? Usually distros target their packages to a single library version, and often people run suites (Gnome, KDE, etc) that use a similar set of libraries in their different processes.

Re: Static Linux

#34

Post by ertdfgcb »

dfc wrote: post #22

> Because dwm is customized through editing its source code, it’s > pointless to make binary packages of it. This keeps its userbase > small and elitist. No novices asking stupid questions. I never understood why the authors of dwm thought this was a "nice" feature of configuration via source code.

They don't mind if DWM is only ever used by a handful of people if only a handful of people agree with the principles behind it.

Re: Static Linux

#35

Post by al2o3cr »

dfc wrote: post #22

> Because dwm is customized through editing its source code, it’s > pointless to make binary packages of it. This keeps its userbase > small and elitist. No novices asking stupid questions. I never understood why the authors of dwm thought this was a "nice" feature of configuration via source code.

I'm gonna go with "because they're dicks".

Re: Static Linux

#36

Post by coherentpony »

zackmorris wrote: post #23

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice.

It works in practice if developers and maintainers adhere to semantic versioning. Unfortunately, there are numerous packages that don't adhere to this standard and that's when widespread breakage occurs.

Re: Static Linux

#37

Post by voltagex_ »

ultramancool wrote: post #17

Cool idea. Not enough people seem aware of executable packers like UPX http://upx.sourceforge.net/ though. These are excellent tools to keep the size down when using large static binaries. By compressing the file on disk and decompressing in memory you often wind up with a smaller and sometimes even faster loading (depending on disk IO speed vs decompression speed) package. I got a static Qt binary from 4 MB down to…

I found a bug with an offline documentation reader (can't remember which) that if you pack one of the QT components (qwindows.dll) the application wouldn't load anymore. Still not sure whether to report a bug and who to report it to.

Yes UPX is cool, but I don't think it's 100% compatible.

Re: Static Linux

#38

Post by chongli »

deathanatos wrote: post #28
zackmorris wrote: post #23

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? Also, a program A depends on library B and library C v1.1. Library B also depends on C, but v1.2. Which gets used? > More often than not, when…

>Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library?

No, this is the responsibility of the server which distributes binaries to users via the package manager.

Re: Static Linux

#39

Post by asuffield »

GeorgeTirebiter wrote: post #32
Lerc wrote: post #20

Earlier quoted context omitted.

The downside to tools like UPX is that the executable code is actively transformed on load. This limits the ability to use shared memory for multiple concurrent executions of the same executable. If the OS loads executables by mmap and load on page-hit, you can potentially save memory by not ever loading unused parts of an executable. a transform-on-load requires the entire program to be loaded before execution begin…

'... never loading unused parts of an executable...' - this is one of the benefits of a paging system. Paging was (is?) the best way to keep memory requirement down. The way Multics worked was by having paged segments. http://www.osinfoblog.com/post/136/segmentation-with-paging:... The x86(-64) can support something like this, but as far as I know, no modern OS supports this feature.

Linux does this, as does basically every other modern OS.

Pick a random large process on a linux host - like your web browser. cat /proc/$pid/status.

VmExe is the size of the mapped executable; VmLib is the size of all the other mapped libraries and executable pages. Add those two numbers together to find the size of all the executable code mapped into this binary.

VmRSS is the amount of physical memory that the process is currently using. You'll find that this is a lot smaller than the code mapped into the binary. That's because the kernel hasn't loaded any of that into physical memory.

Re: Static Linux

#40

Post by [deleted] »

zackmorris wrote: post #23

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

[deleted]
Post reply on HN