Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cosmopolitan Libc: build-once run-anywhere C library

justine.lol

Post reply on HN
Print view View original

31–40 of 171 posts

Re: Cosmopolitan Libc: build-once run-anywhere C library

#31

Post by jart »

groodt wrote: post #23

This is very clever. I wonder if it could be adopted for Python C extensions (where possible) to make it easier to distribute cross-platform binary distributions. Of course it wouldn’t be suitable for gui toolkits, but I imagine that the majority of Python C-extensions could be handled.

Cosmopolitan would actually be a great fit for Python extensions, since manylinux1 requires RHEL5 support and Cosmopolitan is probably the only modern C library that still supports it (glibc these days needs rhel8+). Maybe someone will figure out a way to compile a single-file library that's simultaneously a .dylib / .dll / .so just like how Actually Portable Executable does for programs.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#32

Post by TimTheTinker »

This is an interesting, and potentially compelling argument that WebAssembly binaries and runtime actually aren’t the future of compile-once-run-anywhere systems software: these support syscalls, sockets, and stdin/stdout —- all of which requires platform-specific glue code to achieve using WebAssembly and standard libc (or JS).

Re: Cosmopolitan Libc: build-once run-anywhere C library

#33

Post by BeeOnRope »

jart wrote: post #28
BeeOnRope wrote: post #25

This approach of using inline asm wrappers around functions which specify exactly the clobbered registers is neat, but... I don't think the approach of making a call from inline asm is safe. In particular, it clobbers the redzone [1], which the compiler may have used and expect to be intact after the call. In general, at least some compilers (e.g., gcc) assume there are no calls in an inline asm block. --- [1] https:…

Cosmopolitan doesn't use the "red zone" because these binaries boot on bare metal and operating system code can't assume a red zone. GCC supports this. Otherwise the Linux Kernel wouldn't work, since it's compiled the same way. So invoking call from inline asm is perfectly safe. The only real issue is that the called function can't assume the stack is 16-byte aligned. But that's fine, since Cosmo only using asm(call)…

Well you edited the compile command on the page just now to add -mno-red-zone, but it wasn't there before. So do you mean it is safe as long as -mno-red-zone is used?

Even then, I am not quite sure: redzone is just an example of one of the assumptions that might be violated when calls are made from an asm block, but at least some gcc devs are on record saying that calls from inline asm are not supported: the main problem is that leaf functions can be compiled differently than non-leaf, and a hidden call breaks this (i.e., a non-leaf function may appear as leaf to the compiler).

Re: Cosmopolitan Libc: build-once run-anywhere C library

#34

Post by jart »

TimTheTinker wrote: post #32

This is an interesting, and potentially compelling argument that WebAssembly binaries and runtime actually aren’t the future of compile-once-run-anywhere systems software: these support syscalls, sockets, and stdin/stdout —- all of which requires platform-specific glue code to achieve using WebAssembly and standard libc (or JS).

Author here. WebAssembly has played an important role protecting CI on the Internet and it's cool that it enables us to run LLVM in the browser. I was very surprised when I learned that some people were using it offline outside of browsers, since doing so requires a JVM-like runtime such as wasmtime. Cosmopolitan proves that it's possible to just fix C instead, which might fix all the stuff that's built on top of C too. We can in fact have first-class portable native binaries, which will only require JIT compilation if they're launched on a non-x86 machine. This is probably how Java would have been designed, if Intel's instruction set hadn't been encumbered by patents at the time. That's changed. The x86-64 patents just expired this year. So I say why not use it as a canonical software encoding.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#36

Post by ludocode »

jart wrote: post #19
Sebb767 wrote: post #7

Earlier quoted context omitted.

I can think of two reasons: It's arguably a hack and it's not needed. While this is awesome, it can easily break (zsh already seems to do so) and, as the author states, it is not meant for GUI apps or programs relying deeply on OS APIs, which includes a lot of apps. Additionaly, it does not solve an important problem. Doing a build for each supported OS is not that hard and easily automated. The actual problem is to…

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

> Cosmopolitan only depends on stable kernel ABIs with long term stability promises

Unfortunately, on macOS a stable kernel ABI does not exist. macOS has had kernel ABI breakage before, even recently, and they explicitly warn against making your own system calls. On Windows things are more stable but that's only because Windows cares about backwards compatibility to the point of preserving the behavior of even the most badly written software.

Linux is the only one of the three that promises a stable kernel ABI. On macOS and Windows the system libc is the only safe way to talk to the kernel.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#37

Post by jart »

parhamn wrote: post #35

In theory the linked APE library [1] can also be used for languages like Rust and Go binaries to make fat portable ones as well, right? [1] https://justine.storage.googleapis.com/ape.html

Yes! Cosmopolitan and APE are now permissively licensed under ISC terms. So they'd be a perfect fit for language authors. One of the biggest obstacles language authors encounter, is overcoming the obstacles posed by portability between operating system apis. Cosmopolitan takes care of that. You can just use the normal textbook unix apis and have it work on Windows/Linux/BSDs. Best of all, Cosmpolitan is tiny, embeddable, and unobtrusive. As in, these "fat binaries" are actually more like 16kb in size. So any language author who chooses to use it is going to have more time to focus on their vision, while reaching a broader audience from day one, with a sexy lightweight form.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#38

Post by rurban »

RcouF1uZ4gsC wrote: post #5

> For an example of how that works, consider the following common fact about C which that's often overlooked. External function calls such as the following: memcpy(foo, bar, n); Are roughly equivalent to the following assembly, which leads compilers to assume that most cpu state is clobbered: I think most modern C and C++ compilers know that memcpy is special and optimize accordingly. I believe some C++ compilers rec…

Replace modern with clang, and awful with gcc. I have no idea why gcc is still so fast overall, but its memcpy is horrible, as shown in the blog post. My own memcpy in my libc also beats the gcc/glibc one. The compilers are only good for short memcpy's. For the rest the asm hinders modern compiler optimizations.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#39

Post by moonchild »

jart wrote: post #34
TimTheTinker wrote: post #32

This is an interesting, and potentially compelling argument that WebAssembly binaries and runtime actually aren’t the future of compile-once-run-anywhere systems software: these support syscalls, sockets, and stdin/stdout —- all of which requires platform-specific glue code to achieve using WebAssembly and standard libc (or JS).

Author here. WebAssembly has played an important role protecting CI on the Internet and it's cool that it enables us to run LLVM in the browser. I was very surprised when I learned that some people were using it offline outside of browsers, since doing so requires a JVM-like runtime such as wasmtime. Cosmopolitan proves that it's possible to just fix C instead, which might fix all the stuff that's built on top of C t…

Because x86 instruction encoding is a mess. Take a look at this[1] flowchart from the amd manual (vol. 3).

That doesn't even tell the whole story; there's a ton of path dependence, and the operand format is quite complicated and has tons of special cases.

1. https://0x0.st/-rce.png

Post reply on HN