Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cosmopolitan Libc: build-once run-anywhere C library

justine.lol

Post reply on HN
Print view View original

51–60 of 171 posts

Re: Cosmopolitan Libc: build-once run-anywhere C library

#51

Post by jart »

moonchild wrote: post #46
rurban wrote: post #38

Earlier quoted context omitted.

Replace modern with clang, and awful with gcc. I have no idea why gcc is still so fast overall, but its memcpy is horrible, as shown in the blog post. My own memcpy in my libc also beats the gcc/glibc one. The compilers are only good for short memcpy's. For the rest the asm hinders modern compiler optimizations.

Not sure what your point is calling gcc both ‘awful’ and ‘so fast overall’. Nor why you're conflating gcc and glibc. FWIW my experience has been has clang produces a superior development experience for c++ code, but that gcc is better for c. Llvm does have a more modular architecture than gcc, but it's not really fair to say that it's more modern overall. Gcc has gotten several features ahead of llvm, like hot/cold L…

If anyone's curious, here's the link to memcpy() as it's actually implemented in the Cosmopolitan headers: https://github.com/jart/cosmopolitan/blob/de09bec215675e9b0b... One thing that the web page doesn't mention (for the sake of simplicity) is that the Cosmopolitan headers do call __builtin_memcpy() as well, but only for 2-power constexpr sizes. That's the only time when GCC and Clang both do the optimal thing. In all other cases it's faster to use asm("call MemCpy") which implements something faster than the builtin would otherwise generate. See https://github.com/jart/cosmopolitan/blob/de09bec215675e9b0b...

Re: Cosmopolitan Libc: build-once run-anywhere C library

#52

Post by Someone »

jart wrote: post #34
TimTheTinker wrote: post #32

This is an interesting, and potentially compelling argument that WebAssembly binaries and runtime actually aren’t the future of compile-once-run-anywhere systems software: these support syscalls, sockets, and stdin/stdout —- all of which requires platform-specific glue code to achieve using WebAssembly and standard libc (or JS).

Author here. WebAssembly has played an important role protecting CI on the Internet and it's cool that it enables us to run LLVM in the browser. I was very surprised when I learned that some people were using it offline outside of browsers, since doing so requires a JVM-like runtime such as wasmtime. Cosmopolitan proves that it's possible to just fix C instead, which might fix all the stuff that's built on top of C t…

If Sun would have picked a ‘real’ CPU, I doubt it would have been x86 (did they even sell any of them at the time?)

Also, picking a real CPU is only worth it for running on that CPU, and that particular one. If they had picked x86 in 1996, they wouldn’t even have supported MMX.

In a platform-independent design, using n registers in the virtual CPU is only a good idea for n = 0 (i.e. a stack machine, as in the JVM, .NET CLR and Web Assembly) and n = ∞ (as in LLVM).

If you pick any other number your platform-independent code needs to handle both the case where the number of op real registers is smaller (so you need to have a register allocator) and the case where it is larger (there, you could just ignore the additional registers, but that means giving up a lot of performance, so you have to somehow figure out which data would have been in registers if the CPU had more of them).

Why write and optimize both of these complex problems, if you can pick a number of registers at either end of the scale, and spend all your resources on one of them?

And that’s even more true for x86-64, which doesn’t have a fully orthogonal instruction set, has I don’t know how many ways to do vector instructions, none of which allow you to clearly express how long the vectors you’re iterating over are (making it hard to optimally map them to other CPUs or newer vector instructions), has 80-bit floats, etc.

There’s a reason Google stopped work on https://en.wikipedia.org/wiki/Google_Native_Client in favor of Web Assembly.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#53

Post by moonchild »

jart wrote: post #51
moonchild wrote: post #46

Earlier quoted context omitted.

Not sure what your point is calling gcc both ‘awful’ and ‘so fast overall’. Nor why you're conflating gcc and glibc. FWIW my experience has been has clang produces a superior development experience for c++ code, but that gcc is better for c. Llvm does have a more modular architecture than gcc, but it's not really fair to say that it's more modern overall. Gcc has gotten several features ahead of llvm, like hot/cold L…

If anyone's curious, here's the link to memcpy() as it's actually implemented in the Cosmopolitan headers: https://github.com/jart/cosmopolitan/blob/de09bec215675e9b0b... One thing that the web page doesn't mention (for the sake of simplicity) is that the Cosmopolitan headers do call __builtin_memcpy() as well, but only for 2-power constexpr sizes. That's the only time when GCC and Clang both do the optimal thing. In…

> __builtin_constant_p

Aww, I was hoping you would use the evil ICE_P - https://lkml.org/lkml/2018/3/20/805

Re: Cosmopolitan Libc: build-once run-anywhere C library

#56

Post by rav »

I have Wine installed on my Linux system, which uses "binfmt_misc" to automatically run Windows-looking executable files through Wine. This causes an issue with Cosmopolitan, since it uses an output format that triggers binfmt_misc. I had to run this to disable binfmt_misc on my system:

    sudo systemctl mask proc-sys-fs-binfmt_misc.automount
...and this to disable it for just the current boot:

    sudo bash -c 'echo 0 > /proc/sys/fs/binfmt_misc/status'

Re: Cosmopolitan Libc: build-once run-anywhere C library

#57

Post by Someone »

jart wrote: post #49
moonchild wrote: post #39

Earlier quoted context omitted.

Because x86 instruction encoding is a mess . Take a look at this[1] flowchart from the amd manual (vol. 3). That doesn't even tell the whole story; there's a ton of path dependence, and the operand format is quite complicated and has tons of special cases. 1. https://0x0.st/-rce.png

I thought x86 was pretty straightforward once I understood the octal encoding. Cosmopolitan provides a 3kb x86 instruction length decoder that supports all past, present, and future ISAs: https://github.com/jart/cosmopolitan/blob/master/third_party... The Cosmopolitan codebase also provides a debugger / emulator for i8086 + x86-64 that's similar to GDB TUI and Bochs. See https://justine.lol/blinkenlights/index.html F…

I wouldn’t call 3kb “pretty straightforward”, certainly not relative to the competition. The instruction length decoder for quite a few CPUs is

  return INSTRUCTION_SIZE;
I don’t know any other that has more than a few cases for immediate constants and jump addresses.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#58

Post by jart »

iforgotpassword wrote: post #41
jart wrote: post #19

Earlier quoted context omitted.

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

Why vista only? Is it 64 bit only? Otherwise for TUI-only this should go back to at least NT4. Or is it using anything clever that makes older versions barf? Edit: Ah, maybe it's bypassing ntdll and syscalls changed?

Author here. It's Vista+ because Cosmopolitan focuses on x86-64. Cosmopolitan does however support i8086 since these executables boot from BIOS. You can even define a _start16 entrypoint if you want to write real mode apps. No effort has been made to natively target i386, however it's still technically supported since Actually Portable Executables will re-exec themselves under qemu for other architectures.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#59

Post by modeless »

jart wrote: post #19
Sebb767 wrote: post #7

Earlier quoted context omitted.

I can think of two reasons: It's arguably a hack and it's not needed. While this is awesome, it can easily break (zsh already seems to do so) and, as the author states, it is not meant for GUI apps or programs relying deeply on OS APIs, which includes a lot of apps. Additionaly, it does not solve an important problem. Doing a build for each supported OS is not that hard and easily automated. The actual problem is to…

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

I think the right way to add GUI support to this is to use the web. Simply embed an HTTP server in your binary and open the user's default web browser on launch. I have a framework for this here: https://github.com/jdarpinian/web-ui-skeleton

Re: Cosmopolitan Libc: build-once run-anywhere C library

#60

Post by jart »

rav wrote: post #56

I have Wine installed on my Linux system, which uses "binfmt_misc" to automatically run Windows-looking executable files through Wine. This causes an issue with Cosmopolitan, since it uses an output format that triggers binfmt_misc. I had to run this to disable binfmt_misc on my system: sudo systemctl mask proc-sys-fs-binfmt_misc.automount ...and this to disable it for just the current boot: sudo bash -c 'echo 0 > /p…

Give this a try:

    sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"
That command should allow Actually Portable Executables to co-exist with WINE, which assumes the MZ prefix, since APE binaries always use the longer MZqFpD prefix.
Post reply on HN