Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cosmopolitan Libc: build-once run-anywhere C library

justine.lol

Post reply on HN
Print view View original

41–50 of 171 posts

Re: Cosmopolitan Libc: build-once run-anywhere C library

#41

Post by iforgotpassword »

jart wrote: post #19
Sebb767 wrote: post #7

Earlier quoted context omitted.

I can think of two reasons: It's arguably a hack and it's not needed. While this is awesome, it can easily break (zsh already seems to do so) and, as the author states, it is not meant for GUI apps or programs relying deeply on OS APIs, which includes a lot of apps. Additionaly, it does not solve an important problem. Doing a build for each supported OS is not that hard and easily automated. The actual problem is to…

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

Why vista only? Is it 64 bit only? Otherwise for TUI-only this should go back to at least NT4. Or is it using anything clever that makes older versions barf?

Edit: Ah, maybe it's bypassing ntdll and syscalls changed?

Re: Cosmopolitan Libc: build-once run-anywhere C library

#42

Post by moonchild »

ludocode wrote: post #36
jart wrote: post #19

Earlier quoted context omitted.

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

> Cosmopolitan only depends on stable kernel ABIs with long term stability promises Unfortunately, on macOS a stable kernel ABI does not exist. macOS has had kernel ABI breakage before, even recently, and they explicitly warn against making your own system calls. On Windows things are more stable but that's only because Windows cares about backwards compatibility to the point of preserving the behavior of even the mo…

Ntdll is the kernel ABI on windows; it is stable, and cosmopolitan uses it.

Not sure what you mean by ‘on Windows thing are more stable’; syscall numbers change all the time. See https://j00ru.vexillium.org/syscalls/nt/64/

Re: Cosmopolitan Libc: build-once run-anywhere C library

#44

Post by jart »

smcameron wrote: post #27

> Please note this is intended for people who don't care about desktop GUIs, and just want stdio and sockets [...] Not to diminish the achievement at all, but a significant limitation to be noted (and not a very surprising limitation). I also wonder about things like the sizeof(long). On 64-bit linux, this is 8, but on windows, it's 4. Well, "on windows"... maybe this throws all the APIs that make such assumptions, u…

Author here. That's not an inherent limitation. See https://justine.lol/apelife/index.html for an example of a GUI + TUI that's built using Cosmopolitan. The reason why I said what you quoted, is I'm simply trying to calibrate expectations. I don't view desktop GUIs as a productive area of focus, because there's such a lack of consensus surrounding the APIs that requires, and web browsers do a great job.

As for sizeof(long) it's 64-bit with Cosmopolitan, which uses an LP64 data model. That hasn't impacted its ability to use the WIN32 API. See https://github.com/jart/cosmopolitan/tree/master/libc/nt where WIN32 API functions are declared using uint32_t, int64_t, etc. It works great. I compile my Windows apps on Linux and it feels wonderful to not have to touch MSVC. Cosmopolitan even takes care of the chore of translating your UTF-8 strings into UTF-16 when polyfilling functions like open() which delegates to CreateFile() on Windows.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#46

Post by moonchild »

rurban wrote: post #38
RcouF1uZ4gsC wrote: post #5

> For an example of how that works, consider the following common fact about C which that's often overlooked. External function calls such as the following: memcpy(foo, bar, n); Are roughly equivalent to the following assembly, which leads compilers to assume that most cpu state is clobbered: I think most modern C and C++ compilers know that memcpy is special and optimize accordingly. I believe some C++ compilers rec…

Replace modern with clang, and awful with gcc. I have no idea why gcc is still so fast overall, but its memcpy is horrible, as shown in the blog post. My own memcpy in my libc also beats the gcc/glibc one. The compilers are only good for short memcpy's. For the rest the asm hinders modern compiler optimizations.

Not sure what your point is calling gcc both ‘awful’ and ‘so fast overall’. Nor why you're conflating gcc and glibc.

FWIW my experience has been has clang produces a superior development experience for c++ code, but that gcc is better for c. Llvm does have a more modular architecture than gcc, but it's not really fair to say that it's more modern overall. Gcc has gotten several features ahead of llvm, like hot/cold LTO separation and parallel compilation (the former is now in llvm and the latter apparently is impractical due to architectural problems).

> The compilers are only good for short memcpy's. For the rest the asm hinders modern compiler optimizations

Also not sure what you mean by that. TFA isn't performing memcpys with inline assembly. They're using inline assembly to perform a call to the exact same ‘memcpy’ function you would have called anyway, but also to inform the compiler that memcpy clobbers fewer functions. So it's true that this will mainly improve performance for copies of small memory regions; but the performance of large copies will be unaffected.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#47

Post by jart »

moonchild wrote: post #42
ludocode wrote: post #36

Earlier quoted context omitted.

> Cosmopolitan only depends on stable kernel ABIs with long term stability promises Unfortunately, on macOS a stable kernel ABI does not exist. macOS has had kernel ABI breakage before, even recently, and they explicitly warn against making your own system calls. On Windows things are more stable but that's only because Windows cares about backwards compatibility to the point of preserving the behavior of even the mo…

Ntdll is the kernel ABI on windows; it is stable, and cosmopolitan uses it. Not sure what you mean by ‘on Windows thing are more stable’; syscall numbers change all the time. See https://j00ru.vexillium.org/syscalls/nt/64/

Cosmopolitan uses the stable WIN32 API on Windows. The word ABI is used loosely in this context. It's somewhat accurate since Cosmopolitan still configures the assembler to generate the binary linkage by hand. NTDLL APIs are only used on rare occasions, such as sched_yield(). When it is used, it's used in accordance with Microsoft's recommendations: namely having fallback. It'd be great if we were authorized to use the SYSCALL instruction on Windows, however we have to respect Microsoft's recommendations if we want long-term stability promises.

Mac however is a different story. One of the reasons Apple came back is they adopted the UNIX interface. They literally copied the magic numbers right out of the UNIX codebase. They shouldn't have the right to stand on the shoulders of giants, pull an about face, and then declare it their own internal API. Apple actually broke every single Go application a few years ago by doing that, and I'm astounded that Google didn't do more to push back. We need to give Apple as much feedback as possible about keeping the SYSCALL interface stable. It's the right thing to do.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#48

Post by smcameron »

jart wrote: post #44
smcameron wrote: post #27

> Please note this is intended for people who don't care about desktop GUIs, and just want stdio and sockets [...] Not to diminish the achievement at all, but a significant limitation to be noted (and not a very surprising limitation). I also wonder about things like the sizeof(long). On 64-bit linux, this is 8, but on windows, it's 4. Well, "on windows"... maybe this throws all the APIs that make such assumptions, u…

Author here. That's not an inherent limitation. See https://justine.lol/apelife/index.html for an example of a GUI + TUI that's built using Cosmopolitan. The reason why I said what you quoted, is I'm simply trying to calibrate expectations. I don't view desktop GUIs as a productive area of focus, because there's such a lack of consensus surrounding the APIs that requires, and web browsers do a great job. As for sizeo…

Thanks for the informative reply.

> and web browsers do a great job.

Suppose one had a native linux game that uses POSIX, OpenGL and SDL. Is there any hope?

Re: Cosmopolitan Libc: build-once run-anywhere C library

#49

Post by jart »

moonchild wrote: post #39
jart wrote: post #34

Earlier quoted context omitted.

Author here. WebAssembly has played an important role protecting CI on the Internet and it's cool that it enables us to run LLVM in the browser. I was very surprised when I learned that some people were using it offline outside of browsers, since doing so requires a JVM-like runtime such as wasmtime. Cosmopolitan proves that it's possible to just fix C instead, which might fix all the stuff that's built on top of C t…

Because x86 instruction encoding is a mess . Take a look at this[1] flowchart from the amd manual (vol. 3). That doesn't even tell the whole story; there's a ton of path dependence, and the operand format is quite complicated and has tons of special cases. 1. https://0x0.st/-rce.png

I thought x86 was pretty straightforward once I understood the octal encoding. Cosmopolitan provides a 3kb x86 instruction length decoder that supports all past, present, and future ISAs: https://github.com/jart/cosmopolitan/blob/master/third_party... The Cosmopolitan codebase also provides a debugger / emulator for i8086 + x86-64 that's similar to GDB TUI and Bochs. See https://justine.lol/blinkenlights/index.html For example, here's the ALU code: https://github.com/jart/cosmopolitan/blob/master/tool/build/...

Re: Cosmopolitan Libc: build-once run-anywhere C library

#50

Post by jwilk »

shakna wrote: post #24
modeless wrote: post #13

Earlier quoted context omitted.

A downside of this approach is the binary can't be executed with the exec system call because the kernel doesn't recognize scripts without a shebang. It can only be executed from within a shell.

> A downside of this approach is the binary can't be executed with the exec system call because the kernel doesn't recognize scripts without a shebang. It can only be executed from within a shell. It's a little bit more nuanced than that, depending on the platform. You don't necessarily need to have a shebang. This behaviour is guaranteed in Linux, and traditional on BSD, but not POSIX: > If the header of a file isn'…

This quote is about execlp(), execvp() and execvpe() functions, which are supposed to emulate what shell does.

POSIX has similar wording:

> In the cases where the other members of the exec family of functions would fail and set errno to [ENOEXEC], the execlp() and execvp() functions shall execute a command interpreter and the environment of the executed command shall be as if the process invoked the sh utility using execl() as follows:

  execl(, arg0, file, arg1, ..., (char *)0);
> where is an unspecified pathname for the sh utility, file is the process image file, and for execvp(), where arg0, arg1, and so on correspond to the values passed to execvp() in argv[0], argv[1], and so on.

https://pubs.opengroup.org/onlinepubs/9699919799/functions/e...

Post reply on HN